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PREFACE 


This  report  explores  issues  m  forecasting  and  modeling  the  demand 
for  aircraft  recoveraUe  spare  parts  to  improve  the  Air  Force’s  estima¬ 
tion  of  spares  and  rqmir  requirements  over  quarterly,  annual,  and 
longer  plAnning  horizons.  Specifically,  it  demonstrates  the  utility  of 
eq>proaches  that  account  ezplidtiy  for  nonstationarity  and  their  supe¬ 
riority  over  current  methods  used  by  the  Air  Force  Materiel 
Command  for  these  purposes.  It  is  part  of  a  larger  body  of  research, 
carried  out  in  the  Logistics  Requirements  Prqject,  and  is  intmided  to 
ftnhatiftw  our  understanding  of  the  implications  for  requirements  es¬ 
timation  of  demand  uncertainty  and  logistics  management  adapta¬ 
tions  to  cope  with  it.  The  several  reports  that  describe  this  work  are 
listed  here: 


•  John  B.  Abell  et  al..  Estimating  Requirements  for  Aircraft 
Recoverable  Spares  and  Depot  Repair,  RAND,  R-4210-AF,  1993. 

•  John  B.  Abell  and  Frederick  W.  Finnegan,  Data  and  Data 
Processing  Issues  in  the  Estimation  of  Requirements  for  Aircraft 
Recoverable  Spares  and  Depot  Repair,  RAND,  MR-264-AF 
(forthcoming). 

•  Donald  P.  Gaver,  Karen  E.  Isaacson,  and  John  B.  Abell,  Estimating 
Aircraft  Recoverable  Spares  Requirements  with  Cannibalization  of 
Designated  Items,  RAND,  R-4213-AF,  1993. 

•  Karen  E.  Isaacson  and  Patricia  Boren,  Dyna-METRIC  Version  6: 
An  Advanced  Capability  Assessment  Model,  RAND,  R-4214-AF, 
1993. 

•  John  B.  Abell,  Estimating  Requirements  for  Aircraft  Recoverable 
Spares  and  Depot  Repair:  Executive  Summary,  RAND,  R-4215-AF, 
1993. 

The  first  of  these  reports  describes  the  main  bo^  of  work.  The  sec¬ 
ond  discusses  data  and  data-processing  issues  related  to  estimating 
aircraft  recoverable  spares  and  repair  requirements.  The  third  pre¬ 
sents  a  computational  algorithm  for  estimating  reqiiirements  for  air¬ 
craft  recoverable  spares  based  on  the  assumption  that  items  can  be 
designated  as  cannibalizable  or  not.  The  fourth  describes  the  capabil¬ 
ity  assessment  model  used  to  evaluate  the  stockage  postures  that 
were  anticipated  to  eventuate  fimm  purchases  of  particular  mixes  of 
recoverable  spares.  The  fifth  report  summarizes  the  entire  body  of 

iii 
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wmk  and  discusses  the  mayor  findings  and  recommendations  that 
have  emerged  firtnn  this  research. 

This  work  had  the  joint  sponsorship  of  Headquarters,  United  States 
Air  Force  (AF/LEX);  Headquarters,  Air  Force  Logistics  Command, 
now  tihe  Air  Ftoroe  I^teriel  Command  (AFMCVXP  and  AFMC/XRI); 
and  the  Director  of  Maintenance  Poli^,  Office  of  tiie  Assistant 
Secretary  of  Defense  fi>r  Procurement  and  Logistics.  It  was  carried 
out  in  the  Resource  Management  and  System  Acquisition  Program  of 
Project  AIR  FORCE,  RAND's  federally  fiinded  research  and  develop¬ 
ment  center  (FFRDC)  supported  by  the  U.S.  Air  Force,  and  in  the 
Acquisition  and  Support  Policy  Program  of  the  National  Defense 
Researdi  Institute,  RAND’s  FFRDC  sujqxHrted  hy  the  Office  of  the 
Secretary  of  Ddense.  It  should  be  of  particular  interest  to  those  con¬ 
cerned  widi  spares  and  repair  requirements  estimation,  logistics  qrs- 
tem  design  and  modeling,  and  logistics  poliqr  analysis.  It  should  also 
interest  other  persons  concerned  with  modeling  certain  stochastic 
processes. 


SUMBiART 


This  report  addresses  the  problem  of  estimating  Air  Force  needs  for 
aircraft  reooveraUe  spare  parts  and  tiieir  depot-levd  repair.  Since 
the  forecasting  techniques  imbedded  in  tlm  Air  Force  Materiel 
Command’s  current  requirements  estimatimi  process  were  imple¬ 
mented,  a  great  deal  has  been  learned  about  modeling  parts  demand 
processes^  more  effectively.  The  report  explores  sevc^  issues  in¬ 
volved  in  modeling  and  forecasting  demands  for  aircraft  spare  parts 
along  with  alternative  forecasting  methods  that  can  substantially  re¬ 
duce  e^teeted  predictive  error.  The  researdi  described  hffire  is  part 
a  larger  body  of  work  intended  to  hdp  us  understand  better  the  ef- 
focts  of  unootainty  and  management  adaptations  in  nhaping  the  per- 
fonnance  of  the  logistics  system  in  a  variety  of  peacetime  and 
wartime  scenarios,  and  to  account  explicitty  for  those  effects  in  spares 
and  repair  requirmnents  estimation. 

In  the  last  decade,  logistics  research  at  RAND  has  focused  on  combat 
logistics  sui^ort  This  interest  began  with  particular  attention  to  the 
period  of  transition  from  peacetime  to  wartime  when  activity  levels 
were  anticipated  to  increase  shandy,  thus  dramatically  perturbing 
resource  demand  processes.  That  nonstationarity  in  demand 
prompted  the  exploration  and  devehqimiaat  of  substantial  improve- 
moits  in  logistics  research  tools  and  iq>proaches,  especially  in  the 
Ityna-METRIC  series  of  ctqiability  assessment  mod^  [50-53,  70]. 
Continued  interest  in  the  problems  of  wartime  logistics  suppmt  led 
the  late  Dr.  Gordon  B.  Crawford  to  imdertake  a  project  to  tmderstand 
better  the  foctors  that  caused  particular  recoverable  aircraft  spare 
parts  to  become  ^problem  items.”  It  was  known  popularly  as  the 
"Drivers  Project”  because  of  its  particular  concern  with  factors  that 
tended  to  "drive”  the  performance  of  the  logistics  system. 

One  outgrowth  of  the  Drivers  Project  was  Crawford’s  observation  that 
items  identified  as  problem  items  tended  to  exhibit  high  variability  in 
their  demands.  That  observation  led  him  to  further  explore  and 
quantify  the  magnitude  and  pervasivmiess  ci  varialnlity  in  the  de¬ 
mands  for  aircraft  spare  parts.  Crawford’s  findings  were  puUished  in 


the  Air  Fence’s  Recoverable  ConaomptiQo  Item  Requirements  Rystem,  the  ny*- 
tem  used  to  compute  qwres  and  repair  requiremmita,  dtmaiuU  are  defined  as  reraov^ 
of  components  firom  tteir  next  hi^nr  assemblies,  wrehuHiip  components  that  mn  de¬ 
clared  to  be  serviceable  after  subsequent  bench  dieek.  They  also  exclude  part  rsmovals 
to  fimbtate  other  maintenance,  etc. 
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January  1988  [2],  but  the  subatance  of  the  work  was  finished  in  1985 
and  helped  motivate  the  Uncertainty  Project,  a  meyor  research  effort 
at  RAND  that  explored  the  effectiveness  of  management  adaptations 
in  coping  with  uncertainty,  espedally  in  wartime.^  The  adaptati<ms 
iiududed  cannibalization;  late^  supfdy;  lateral  repair,  priority  re¬ 
pair;  assured,  respmisive  intertheater  and  intratheater  transporta¬ 
tion;  and  responsive  depot  repair.  The  importance  of  such  adapta¬ 
tions  was  illuminated  by  Cohen,  ^beU,  and  lippiatt  [20]  in  their 
summary  of  the  mcgor  evaluations  and  underlying  logic  of  the 
Uncertainty  Project. 

Hie  thrust  of  the  forecasting  work  described  here  is  a  different  ap- 
proadi,  in  a  sense,  than  our  recent  past  work,  althou^  it  extends  a 
larger  bo^y  of  demand  modeling  woHe  done  at  RAND  in  the  1950s  and 
1960s,  and  reopened  Muckstadt,  Crawfind,  and  Carrillo  [2,  49,  55] 
in  the  last  decade  or  so.  The  methodtdogy  that  emerged  firom  this 
particular  part  of  the  current  project  is  a  potentiaUy  usefiil  and  im¬ 
portant  improvement  to  the  current  system’s  approach  to  demand 
modeling  and  forecasting. 

This  wmk  does  not  resolve  the  important  problem  of  estimating 
wartime  dmnand  rates.  Althoui^  the  mix  of  demand  rates  resulting 
firran  the  undertying  failure  process  may  not  necessarily  change  dra¬ 
matically  in  wartime,  many  of  the  events  that  can  occur  in  wartime 
are  essoatially  unknowable  in  advance.  Better  demand  fisrecasting 
tiiat  incorpmrates  Bayesian  updating  may  be  helpful  in  attyisting  to 
dianges  in  wartime  more  quickly  than  the  current  system  would  but, 
dearly,  improving  our  demand  forecasting  ability  cannot  1^  itself 
hdp  us  know  the  unknowaUe. 

The  current  system  uses  sm  ei{fot-quarter  moving  average  of  past  de¬ 
mands  by  line  item  to  estimate  item  demand  rates.  This  approsufo 
assigns  as  much  importance  to  e^^ts  in  the  more  distant  past  as  to 
recent  events.  Moreover,  in  forecasting  future  demands,  we  currently 
assign  no  more  imcertainty  to  events  far  in  the  future  than  we  do  to 
events  in  the  short  term.  Our  models  smsume  that  parts  demand  pro¬ 
cesses  have  certain  characteristics  which  empirical  observations  tell 
us  they  do  not  have.  For  example,  in  general,  demand  inrocesses  are 
nonstationaiy;  we  assume  stationarity.  Th^  are  not,  in  general. 


*nie  Uacartsinty  Pwjeet  wtu  psrt  rfRAMya  Resourca  MsnaeBincnt  Program.  It 
was  fyeamOx  eatitM  “Bdumefaic  the  Integritloa  and  ReqMoaiveBeas  of  the  LogistiGS 
S«9|M*t  Qjrrtem  to  Meet  Paawitime  ud  Warttme  Ubcertaintiea”  and  wae  eponaored  ty 
Headsoartate,  USAF/LEX,  and  Headsoartm,  AFLC/XP. 
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compoand  Pouson  processes.^  The  large  variabilily  we  obeerve  in 
theae  processes  is  typically  not  due  to  batdiing,  or  aanpounding,  as  is 
so  often  assumed  in  the  bterature  and  in  our  mod^  ci  dwmand- 
Certainly,  nonstationarity  pla3rs  an  important  role  in  shaping  the 
variability. 

Important  characteristiGS  of  the  statistic  used  by  AFMC  to  estimate 
the  variance-to-mean  ratio  (VTMR)  of  parts  demand  processes  are 
also  discussed  here.  The  variance  of  the  VTMR  estimator  increases 
as  a  function  of  the  coarseness  of  the  partitioning  of  the  observed 
data,  even  when  the  demand  proems  is  stationary.  If  the  process  is 
nonstationaiy,  the  expected  value  of  the  estimator  and  its  variance 
increase  with  the  coarseness  of  the  partitioning  and  with  the  demand 
rate,  behavior  that  is  consistent  with  the  assodatiofi  of  Yd^  values  ci 
ftte  VTMR  with  hi{d^  values  of  demand  rate.  These  findings  suQiest 
that  there  are  estimation  problems  associated  with  AFMCs  use  of  the 
VTMR  estimator.  In  this  report,  we  present  an  improved  approach  to 
specifying  the  variance  that  has  more  satisfying  properties  than  the 
current  model. 

An  approach  to  demand  forecasting  that  seems  especially  appealing 
on  an  intuitive  level,  and  that  performs  well  in  empirical  evaluations, 
is  wei^ted  r^^ression,  a  special  case  of  the  Kalman  filter.  It  is  a  log¬ 
ical  extension  to  Bayesian  statistics  tiiat  explicitly  accounts  for  non- 
stationarify  in  stoduastic  processes,  assigning  greater  wei^t  to  more 
recent  past  demands  than  earlier  ones.  Coupled  with  the  improved 
approa^  to  variance  estimation  wfaidi  assigns  greater  uncertainfy  to 
longer  planning  horizons  than  to  shorter  ones,  it  holds  the  promise  of 
reducing  the  cost  of  spares  investments  while  achieving  adequate  lev¬ 
els  of  system  performance.  For  planning  horizons  10  to  13  quarters 
long,  the  improved  techniques  reduced  forecasting  errors  on  hij^ 
demand  items  by  roug^y  40  to  50  pmreent,  as  shown  in  Table  S.l. 
(Also  see  Figures  6.3  and  5.4.) 

Table  ai 

Percentage  Improvement  in  Root  Mean  Stpiared 

Error  and  Mean  Abedute  Deviation  of  Improved 
Tedbnhpies  over  Current  System 


Meeoun 

48 

38 

MAD 

51 

45 

drfinitioii  of  a  oomiMMind  Pnaoon  proeett  may  bo  finmd  in  the  Gloaauy . 
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Moreover,  in  the  case  ai  the  requirements  cemputation  done  with  the 
March  1966  D041  databaae,  about  $76  miUion  in  procurements  of 
primary  (qwrating  stock  could  have  been  saved  wi^  the  improved 
demand  ami  variance  forecasting  techniques  in  {dace  with  a  modest 
improvement  in  system  performance,  as  shown  in  Table  S.2. 

The  improved  terliniqMew  also  enabled  an  investment  reduction  oi  sl- 
most  a  quarter  of  a  billion  drilars  ($239  million)  with  rou^y  the 
same  perfmrmanoe. 

Ihese  results  are  shown  in  Table  S.3.  (Hie  details  of  these  evalua¬ 
tions  are  explained  in  Section  7.) 

In  the  ai^cations  reported  here,  the  improved  demand  forecasting 
method  incorporates  a  normal  dmtribution  to  approximate  quarterly 
demand.  Compared  to  the  current  system,  it  performs  especially  well 
on  items  with  a  mean  of  15  m  more  demands  per  quarter  (hi^- 
demand  items),  yiriding  the  rou|d>ly  40  to  50  percent  reduction  in 
RMSE  and  MAD  already  mentioned.  We  do  not  recommend  its  use  on 
low-demand  items.  Its  performance  on  low-demand  items  was  not 
impressive,  perhaps  due  to  failure  of  its  underlying  assumptions. 

Table  a2 

Coot  and  Perfocmaaoe  with  Traditional  AvailaUlity  Goals 

Pwcentage  of  Auvraft  Unavailable, 

I  _ Peace  time _  j 

i  Current  Oystem,  Improved  Methods, 

I  Manafement  Adaptations  $3,709  Million  $3,633  Million 

Nocannibalisation  74.9  71.7 

Full  cannihaHsatkm  33.0  32.3 

Cannibalization,  lateral  siqiply  17.3  16.4 

Cannibalization,  quidt,  late^  supply _ 3^2 _ 3^1 _ 

TaUeS,8 

Coot  and  Performance  with  Reduced  Budgets 

Percentage  of  Aircraft  Unavailable, 

_ Peacetime _ 

Current  Syatem,  Improved  Methods, 

_ Management  Adaptationa _ $3,474  Million _ $3,470  Million 

No  cannibalization  81.6  76.3 

FUl  cannibalization  34.6  33.1 

Cannibalizatkm,  lateral  supply  19.0  17.2 

Cannibalization,  quick,  lateral  suiqily  3.6  3.6 


IX 


As  the  literature  suggests,  estimating  demand  rates  on  low-demand 
itmns  has  always  been  a  troublesome  topic,  and  this  work  does  not  re¬ 
solve  the  problem.  The  idea  of  pooling  ^ta  across  many  low-demand 
items  to  gain  strength  from  acklitional  observations  in  estimating 
demand  rates  for  individual  items  was  suggested  years  ago  in  earlier 
RAND  work.  The  idea  seems  worth  pursuing  in  future  research. 

Pot  hi^-demand  items,  we  recommend  using  a  wei^ted  regression 
technique  for  demand  forecasting  and,  for  aU  items,  an  improved 
method  for  specifying  the  VTMR  of  the  probability  distribution  used 
to  describe  the  numbers  of  assets  of  each  type  in  resupply  (i.e.,  in  the 
pipeline).  The  weighted  regression  technique  is  a  more  easily  imple¬ 
mented  version  of  the  general  Kalman  filter  model,  especially  for  to¬ 
tems  as  large  as  the  Air  Force’s  recoverable  item  inventory  system. 
Implementing  these  techniques  should  reduce  the  investment  re¬ 
quired  to  achieve  a  specified  level  of  aircraft  availability  in  peacetime 
a  substantial  amount. 
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GLOSSARY 


A  priori.  Witliout  prior  infinrmatioiL 

AF AO.  Authorized  farce  aequirition  objective.  The  total  uumbor  of 
assets  that  the  spares  requirements  computation  specifies  feu  the  en¬ 
tire  inventory  system. 

AFLC.  Air  Force  Logistics  Command,  now  the  Air  Fmve  Materiel 
Command. 

AFMC.  Air  Force  Materiel  Cmnmand,  fiamerly  the  Air  Fcarce  Logia- 
tics  Command. 

Aircraft  Availability  Model  (AAM).  The  tystem  of  software 
imbedded  in  D041  that  is  used  to  compute  requirements  fm  safety 
stodk  of  sriected  recoverable  items. 

AOCP.  Aircraft  out  of  commission  for  parts.  In  earlier  years  in  the 
Air  Force,  aircraft  were  AOCP  when  they  couldn’t  safely  be  flown  be¬ 
cause  of  lack  of  parts. 

Bayeaiaii.  A  term  deriving  £rom  the  work  of  Bev.  Thomas  Bayes 
(1701-1761)  describing  an  approach  to  optimal  learning  combining 
new  data  with  prior  judgments  or  old  data  using  the  laws  of  condi¬ 
tional  probability. 

Beddown.  A  term  denoting  the  aDocation  of  weapons  by  type  to  lo¬ 
cations. 

Bias.  The  property  of  a  statistical  estimator  such  that  its  mathemat¬ 
ical  expectation  differs  from  the  numerical  value  of  the  parameter  it 
is  used  to  estimate. 

BP15.  Budget  Program  15,  a  category  of  appropriated  fimds  allo¬ 
cated  to  recoverable  aircraft  replenishment  spares. 

Compound  Poisson  process.  A  stochastic  process  in  which  the 
numbers  of  arrivals  that  occur  in  dierjoint  time  intervals  of  equal 
length  are  described  by  the  Poisson  probability  distribution,  and  the 
number  of  events  that  occur  with  each  arrival  is  described  by  a  sepa¬ 
rate,  usually  different,  probability  distribution.  The  number  of  events 
that  occur  with  each  arrival  is  c^ed  the  compounding  random  vari¬ 
able. 

Consumable.  The  property  of  a  part  or  material  such  that  it  is  dis¬ 
carded  after  failure  or  is  consumed  in  use. 


CONUS.  The  continental  United  States. 


C£98.  The  Central  Secondary  Item  Stratification,  AFMC’s  system 
that  is  incmrporated  in  D041,  along  with  the  Aircraft  Availability 
Mo(tel,  for  computing  requirements  for  recotveraUe  spares. 

DOM.  AFMCs  Central  Stodc  Levding  l^rstem  that  allocates  stodc 
levels  for  recover aMe  items  to  the  bases  and  the  depot 

D041.  AFMC’s  system  for  cmnputing  requiremmits  for  aircraft  v- 
erable  spares,  formally  entitled  the  Recoverable  Consumption  item 
Requirements  System. 

Degenerate  iwrohaiiiM^  diatribation.  A  probability  distribution  of 
a  random  variable  with  only  one  possible  value. 

Dwmiwd.  In  the  Air  Force’s  RecoveraMe  Consumption  Item  Require¬ 
ments  System,  the  system  used  to  compute  spares  and  repair 
requirements,  demands  are  defined  as  removals  vS  components  fitna 
their  next  hi^er  assemblies  excluding  components  that  are  declared 
to  be  serviceaUe  after  subsequent  bench  check.  They  also  exclude 
part  removals  to  facilitate  other  maintenance,  etc. 

Emj^ricad  Bayes  iwocediires.  Bayesian  procedures  in  which  ob¬ 
served  data  are  used  to  estimate  the  prior  distribution  in  lieu  of  sub¬ 
jective  judgment. 

EOQ.  Eoon<miic  order  quantity,  requisition  quantity  that  is  de¬ 
termined  to  be  the  most  cost-^ective,  usually  a  function  of  demand 
rate,  reorder  cost,  holding  cost,  interest  rate,  and  unit  price. 
Formulations  based  on  shortage  cost  are  also  common. 

E^wnential  smoothiiig.  A  procedure  for  discounting  observations 
more  heavily  the  further  they  occurred  in  the  past  hx  multiplying 
each  precessive  observation  by  an  increasing  integral  power  of  a 
number  between  zero  and  <me. 

Gamma  probalriUty  distribatioii.  A  probability  distribution  of  the 
continuous  type  whose  density  function  is  given  by 
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Homogeneous  process.  A  stodiastic  process  whose  probability  dis- 
tributiim  is  invariant. 

Jaama-8toin  estiauitora.  A  dass  of  empirical  Bayes  estimators 
formed  fay  moditying  estimates  vS  parameters  underlying  observations 


of  realizati<m8  of  a  stochaatic  process  Iqr  pooling  them  with  observe- 
ti<m8  from  other,  aimilar  populations. 

LRU.  liine-replaoeable  unit,  a  part  or  assembly  that  is  typically  re¬ 
moved  directly  from  an  aircndt  when  undergoing  maintenance  other 
than  adjustment,  calibration,  or  smvidng. 

MAD.  Mean  absolute  deviation.  The  average  unsigned  difference  be¬ 
tween  a  set  of  estimators  and  the  true  values  of  the  parameters  being 
estimated. 

Master  stock  number.  The  rtock  number  assigned  to  the  preferred 
item  in  a  set  of  two  or  more  interchangeaUe  items. 

METRIC.  Multi-Echelon  Technique  fin:  Recoverable  Item  Control,  a 
method  for  estimating  requirements  fior  aircraft  recoverable  spare 
parts  developed  by  C.  C.  Shertxtxdce  of  RAND. 

MICAP.  A  term  denoting  mission  capability  effect  of  a  part  shortage. 

Moving  average.  The  statistic  formed  by  the  mean  of  a  fixed  num¬ 
ber,  n,  of  observatimis  of  a  stochastic  process  where  the  most  recent  n 
observations  are  summed  and  divided  by  n.  As  observations  accumu¬ 
late  over  time,  the  latest  observation  is  added  to  the  sequence  and  the 
n  -I-  1st  observation,  counting  badcward,  is  discarded. 

Negative  binomial  probability  distribution.  A  probability  distri- 
bution  of  the  discrete  type  that  may  apply  to  situations  in  which 
events  occur  at  random  but  the  variance  of  the  numbers  of  events  in 
nonoverlapping  time  intervals  of  equal  length  is  higgler  than  allowed 
by  the  Poisson  distribution.  Its  density  fiinction  is  given  by 

NRTS.  Not  repairable  this  station.  The  designati<m  is  given  to  a  re¬ 
pairable  part  whose  repair  is  beyond  the  capalnlity  of  maintenance  at 
a  particular  location. 

Partitioning.  The  subdivision  of  a  set  into  subsets  that  are  mutu- 
albr  exclusive  and  collectively  exhaustive. 

Poisson  process.  The  most  widdy  known  and  often  used  finrm 
stochastic  model  with  important  mathematical  properties  that  make 
it  especially  tractable  and  usefiiL  It  is  described  by  the  Pdsson  prob- 
alnlity  distribution  whose  density  fiinctian  is  given  by 


POM.  Plrogram  OfajectiviM  Memoraiiduin,  the  programming  docu¬ 
ment  used  by  the  military  departments  to  state  their  budgetary  re¬ 
quirements  in  future  fiscal  years. 

POfiL  Primary  <q>erating  stock,  formerly  known  as  peacetime  operat¬ 
ing  stock. 

Power  ftuietiaii.  A  mathematical  function  of  the  form  y  s  ax^. 

OPA.  Quantity  per  apidication,  the  number  of  parts  of  a  particular 
type  ins^ed  on  the  part’s  next  hij^ier  assembly. 

Baiidom  walk.  A  stodiastic  process  whose  location  parameter 
varies  firom  one  realization  to  the  next  in  a  manner  determined  by 
chance. 

MBAIiM.  The  Readinesa^ESxecution  Availability  Logistics  Module,  the 
s(rftware  module  of  WSMIS  that  computes  requirements  for  war 
readiness  spares. 

BeooveraUe  parts.  A  dass  of  parts  that  are  repaired  when  thqy 
fiul,  rather  than  being  discarded  or  consumed  in  use. 

Rasapirfy.  The  state  of  parts  that  are  in  base  repair,  depot  repair, 
shiimient  fimn  one  location  to  another,  or  have  been  condemned  and 
wb^  replacement  is  pending. 

BM8E.  Root  mean  squared  error,  computed  as  the  square  root  of  the 
average  squared  dUIiBrenoe  between  a  set  of  estimators  and  the  true 
values  of  the  parametws  being  estimated,  a  popular  measure  of  pre¬ 
dictive  aocursf^.  Also  referred  to  as  RM8P,  root  mean  squared  devia- 
ti<m. 

Gtafety  ttoek.  Spares  authorized  to  aoonnmodate  the  variability  in 
the  numbers  of  Hems  in  resupi^. 

fitowneea.  The  characteristic  of  a  jnobability  distribution  such  that 
H  is  atymmetrical  about  the  mean. 

Spectral  aaatyaia.  A  method  for  finding  and  quantifying  periodidty 
in  data. 

SKU.  Shqp-reidaoeable  unit,  a  subassemMy  of  an  LRU  that  is  typi¬ 
cally  rq>laoed  during  rquor  of  the  LRU. 

Stuthmazy  process.  A  stochastie  process  whose  parameters  are  in¬ 
variant. 
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Stodktov^  Servioeables  on  hand  plus  due-ins  minus  due-outs. 

VSL.  VariaUe  safety  level,  a  spares  requiremmits  estimatiim  method 
derived  directly  from  METRIC,  a  method  developed  by  C.  C. 
Sherbrooke  of  RAND. 

VIMR.  Varianoe-to-mean  ratio,  defined  as  the  unbiased  estimator 
the  variance  of  a  process  divided  by  its  mean. 

WRSK.  War  readiness  spares  kit,  an  airliibdeployable  set  of  spares 
to  support  squadrons  deplt^ed  in  contingencies. 

WSMIS.  AFMC’s  Weapon  System  Management  Informatimi  System. 
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1.  INTRODUCTION 


This  report  addresses  the  difficult  proUem  ci  forecasting  Air  Finee 
requirements  for  aircraft  spare  parts  and  their  depot-level  repair. 
These  forecasts  are  made  over  difformt  time  horinms  accordiiig  to  ffie 
specific  purpose  of  the  forecast.  For  example,  the  Air  Force  is  re¬ 
quired  to  forecast  its  budgetary  requirements  for  these  resources 
across  the  multi-year  hotixtms  invdved  in  the  POM.  AFMC  also  fma- 
casts  annual  spares  requiremmits  and  annual  and  quarteriy  repair 
requirements.  Hie  difficulty  of  making  such  forecasts  has  two  fun¬ 
damental  nxrts:  (a)  substantial  varial^ty  in  spares  demands,  even 
in  peacetime  (statistical  unoortointy),  and  (b)  instability  in  force 
structure,  finroe  beddown,  flying  hour  programs,  funding  profiles,  itmn 
reUalnlities,  and  othmr  item  characteristies  (stata-(tf-the-world  uncm*- 
tainty). 

The  levds  of  variability  in  peacetime  dmnands  wiU  probal^  be  com¬ 
pounded  in  wartime  by  system  disruptions,  resource  losses,  and  the 
inevitable  surprises  of  combat,  mcacerbating  the  demand  forecasting 
problem.  (The  estimation  of  wartime  demand  rates  is  not  addressed 
in  this  work.)  Tbe  robustness  of  the  spares  postures  delivered  by  al¬ 
ternative  aiiproadies  to  spares  and  repair  requirements  estimation,  a 
central  issue  of  the  larger  body  of  research  of  whidh  this  is  (me  part,  is 
reported  elsewhere,  as  mentioned  in  the  Preface.  RAND  has  devoted 
considerable  attention  in  recent  years  to  the  role  of  management 
adaptations,  e.g.,  cannibalization,  lateral  supply,  and  priority  repair, 
in  overcoming  uncertainties  in  rescmrce  demands  in  peacetime  and 
wartime.  That  research  and  its  relationship  to  tiiis  work  are  dis¬ 
cussed  briefty  in  Sectkm  2. 

It  is  difficult  to  make  accurate  forecasts  of  spares  and  repair  reciuire- 
ments.  On  more  than  one  occasion,  the  Air  Force  has  had  to  a(liust 
its  budgetaiy  requirements  for  these  resources  as  the  execution  year 
approached,  inducing  considerable  turmoil  in  the  resource  allocation 
process,  lippiatt  noted  that  current  recjuirements  and  capability  as¬ 
sessment  ^}rstmns  do  not  explicitly  consider  parameter  variabilities 
and  finrecasting  uncertainties  [1,  p.  vi].  The  forecasting  algorithm  of 
AFMCs  spares  and  repair  recjuirmnents  estimation  qrstem  can  be 
improved.  The  current  system  usm  ei^t-quarter  moving  averages  to 
estimate  item  demand  parameters,  a  technique  that  gives  no  more 
wei^it  to  relative  recent  observations  than  to  (dder,  (dten  less  rde- 
vant  (foeervatkms.  The  estimates  that  emerge  firmn  this  approach  do 
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not  reflect  the  mmstationarity  commonly  observed  in  these  demand 
processes.  Moreover,  the  current  system  assumes  a  strictly  linear  re¬ 
lationship  between  demands  and  flying  hours.  Thus,  if  the  planned 
flying  hour  program  increases  by  10  percent,  the  system’s  estimates 
of  eq>eeted  dananda  also  increase  10  percent.  Unpublished  analysis 
suggests  that  the  linearity  assumf^km  is  not  only  incorrect,  under 
some  conditions  it  is  grossly  incorrect  in  attributing  too  large  an  efiect 
to  flying  hour  changes. 

Crawflnrd  duuecterized  the  magnitude  aiui  pervasiveness  of  varial^- 
ity  in  peacetime  demand  [2].  He  noted  that,  in  estimating  war  readi¬ 
ness  spares  requirements  and  in  capability  assessment  modeling, 
levds  of  variability  in  demand  were  typically  understated.  In  unpub¬ 
lished  researdb,  he  also  noted  that  dbeerved  demands  wmre  not  lin¬ 
early  rdated  to  ^ying  hours;  he  estimated  a  mudi  smaller  effect  of 
ftying  hours  on  demands.^  Ilius  assumptions  of  the  current  sys¬ 
tem  are  in  smious  doubt  Crawford  hypothesized  that  the  use  of  the 
Kalman  filter,  coupled  with  an  improved  model  of  the  relationship  be¬ 
tween  demands  and  flying  hours,  mi^t  be  a  more  appropriate  ap¬ 
proach  to  modeling  demands  for  aircraft  spare  parts.  The  research 
reported  here  examines  and  confirms  Crawford’s  hypothesis  about  the 
Kalman  filter  and  recommends  the  use  of  a  special  case  of  the 
Kalman  filter,  wei^^ted  repression  forecasting,  for  estimating  spares 
demands.  The  techniques  developed  here  can  be  e:q)ected  to  achieve 
specified  levels  of  system  performance  at  less  cost  throu|^  more  reli¬ 
able  demand  forecasting. 

In  Secticm  2  we  discuss,  by  way  of  badcgnmnd,  the  larger  context  of 
this  research,  its  relation^p  to  past  RAND  research,  and  AFMCTs 
current  approadi  to  modding  dmnands  in  its  estimation  spares  and 
repair  requirements.  We  describe  some  base-level  demand  experience 
in  Section  3  and  discuss  its  implications  for  demand  modeling,  spares 
and  repair  requirements  estimation,  and  inventory  managmnent.  In 
Section  4  we  provide  an  elementary  description  of  the  Kalman  filter 
and  its  historical  derivation  and  describe  our  formulation  and  evalua¬ 
tion  oi  alternative  approaches  to  demand  forecasting.  We  describe 
our  evaluations  of  some  alternative  demand  finrecasting  techniques  in 
Secti<m  5,  including  amig^ited  regression  forecasters,  which  are  a 
special,  limiting  case  of  Kalman  filter  regression  forecasters.  We  dis¬ 
cuss  an  alternative  model  of  variance  specification  in  Section  6  and 

^In  a  eonvmatioii  viOi  ana  af  fha  anttwra  diartty  baflna  hia  death,  Crawfbrd  ra- 
ported  aapieimg  a  paahilatad  model  af  the  forai  h>D-a->blnH,  whMV  D  ia  demanda 
andHflyiiighaara.  Hia  aaUmate  of  the  value  of  the  ooeffident  b  waa  0.2.  To  the  beat 
of  our  knovdedce,  me  eathaate  waa  made  with  F-16  data. 
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presoat  our  evaluatiooB  of  an  improved  «temand  forecasting  method 
and  an  alternative  specification  of  variance  in  Section  7  using  a  capa¬ 
bility  assessment  model  and  replicas  of  the  Air  Force’s  spares  re¬ 
quirements  cmnputation  and  ass^  allocation  ^sterns.  We  offer  some 
Mef  concluding  remarks  and  recommendati<ms  in  Section  8. 


2.  BACKGROUND 


Foot  several  years  beginning  in  the  inid-19508,  RAND  researchers 
pursued  the  problem  of  fcnrecasting  demands  for  aircraft  spare  parts. 
That  interest  was  sustained  throuj^  1969  when  George  Fishman 
puMished  a  Research  Memorandum  (RM)  on  improved  forecasting 
methods  and  thus  ended  for  more  than  a  decade  the  publication  of 
any  research  on  demand  forecasting.  The  next  publication  in  the 
RAND  literature  on  the  topic  appeared  in  1980.  It  addressed  the  is¬ 
sue  of  nonstationarity  in  spare  parts  demand  processes,  a  topic  that 
absorbed  considerable  attention  among  RAND's  logistics  researchers 
throu^out  the  1980s.  Two  recent  publications  have,  in  an  important 
sense,  synthesized  the  principal  Hurusts  of  the  thinking  of  RAND  re¬ 
searchers  over  these  four  decades.  We  will  summarize  them  in  due 
course.  It  is  interesting  to  note  that  this  current  work  has  roots  in 
the  earlier  RAND  research  of  the  1950s  and  1960s  as  well  as  that  of 
the  1980s.  In  the  pages  that  follow,  we  summarize  the  work  at  RAND 
with  special  emphasis  on  those  issues  that  relate  most  directly  to  our 
current  thinking  about  this  important  topic. 

THE  EARLY  YEARS:  1954-1969 

RAND’s  earliest  work  in  demand  forecasting  was  published  in  July 
1954.  It  was  made  possible  by  a  special  data-collection  effort  that 
provided  RAND  resesirchers  with  about  1,300  aircraft  months  of  data 
describing  demands  for  spare  parts  at  three  B-47  bases.  That  particu¬ 
lar  database  absorbed  the  attention  of  several  researchers  for  several 
months.  It  contained  spares  demands  generated  by  about  230  tiircraft 
and  33,000  flying  hours.  Before  the  special  data-collection  project 
that  generated  these  data,  data  describing  demands  by  "aircraft 
modd”  (weapon  system)  had  been  unavailable.  Only  data  describing 
issues  of  spares  had  previously  been  available  but  not  by  weapon  sys¬ 
tem,  only  by  "pnqierty  class”  (federal  stock  dass)  that  overlapped 
weapon  system.  Issues,  of  course,  did  not  reflect  backorders',  thus,  in 
these  early  years,  analysts  were  somewhat  constrained  in  their 
ability  to  identify  and  characterize  the  nature  of  true  demands, 
(jeisler.  Brown,  and  Hizon  [3]  summarized  the  earliest  analytic 
findings  based  on  the  B-47  data  as  follows: 

It  is  shown  that  there  was  a  surprisingly  low  amoimt  of  demand  both  as 
to  kinds  and  quantity  of  aircraft  spare  items,  at  either  MacDill,  March 
or  Faiifind  (En^and)  Air  Bases,  compared  with  the  number  of  kinds  of 
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such  items  in  the  United  States  Air  Force  (USAF)  Stq>ply  Catalogue, 
and  the  quantities  of  such  items  stocked.  Furthemuffe,  we  could  find 
no  significant  correlation  between  the  kinds  or  quantity  oi  items  de- 
man(M  and  the  flying  activity  of  the  aircraft,  measured  in  either  flying 
hours,  landings  or  aiicraft  months.  We  also  found  that  a  comparatively 
small  per  cent  of  the  reparable  tum-ins  airframe  and  engine  spare 
items  w«re  repaired  at  the  rate  base  studied  (MacOill  Air  Force  Base) 
with  most  of  die  items  either  condemned  at  the  base  or  sent  to  the  de¬ 
pot  for  repair. 

These  conclusions  are  hardly  surprising  in  retrospect  because  the 
B-47  database  induded  both  consumables  and  recoverables.  It  is  im¬ 
portant  to  note,  however,  the  lack  of  “significant  correlation”  between 
demands  and  activity  levels.  Shortly  after  this  first  publication. 
Brown  and  Geisler  [4]  reported  their  analysis  of  demand  data  for  B-47 
airfi-ame  parts: 

Both  the  daily  demand  for  individual  kinds  of  items  and  the  daily  com¬ 
bined  demand  over  all  items  show  more  variation  than  expected  from 
the  Poisson  distribution,  which  was  used  as  the  theoretic^  model  of 
demand.  .  .  .  These  residts  indicate  that  if  the  Poisson  distribution  is 
used  to  represent  the  demand  pattern  for  spare  items  because  of  its 
mathematical  convenience,  the  actual  distribution  for  either  individual 
items  or  combined  may  be  more  extreme,  in  that  the  variance  of  the  dis¬ 
tribution  wiU  be  greater  than  the  mean  i^ue  of  demand. 

Thus,  it  was  recognized  finm  the  start  that  demands  for  aircraft 
spares  exhibited  unexpectedly  high  variation.  Again,  the  authors 
failed  to  difierentiate  between  consumables  and  recoverables;  how¬ 
ever,  they  did  observe  differences  in  demand  patterns  between  high- 
and  low-cost  items. 

Geisler  and  Youngs  [5]  explored  one  implication  of  the  B-47  demand 
data  analyses  for  base  supply  stockage  policy.  They  suggested  that 
expensive  items  with  low  demand  rates  shoidd  not  be  stocked  at  the 
base.  Thus  the  use  of  cost  as  well  as  demand  as  a  criterion  for  stock- 
age  also  emerged  rif^t  finm  the  start.  Geisler  and  Youngs  made 
other  fimdamentally  important  observations  in  this  early  paper. 
They  discuss  at  some  length  the  problem  that  past  demand  may  not 
shed  much  li^t  on  an  item’s  “true”  demand  rate.  They  say,  “In  point 
of  fact,  one  of  the  most  troublesome  problems  is  that  of  obtaining 
some  grip  on  the  true  demand  rate.  I^s  is  particularly  true  of  the 
host  of  items  (10,000  strong)  in  the  OlA-FE  category  which  have 
never  been  demanded”  [emphasis  added].  The  idea  that  items  have 
nonstationaiy  demand  rates  does  not  emerge  until  considerably  later. 
After  exploring  the  implications  of  the  decision  to  stock  an  item  with 
low  or  no  observed  demand  in  some  past  period,  the  authors  point  out 
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that  *88  a  matter  of  general  policy  it  may  be  ai'gued  that  the  decision 
on  whether  these  expensive  items  are  to  be  stocked  should  not  be  un¬ 
dertaken  at  base  level  where  the  data  are  not  extensive  (and  there- 
fore  sulgect  to  more  variations)  in  comparison  witii  the  data  at  depot 
level,  llus  question  is  much  affected  Iqr  the  stockage  polity  assumed 
by  the  depot  when  it  determines  the  worldwide  requirements  and  es- 
t^lishes  procurement  needs.”  This  observation  seems  consistent 
with  the  Air  Force’s  later  implementation  of  central  stock  leveling. 
Ironically,  however,  the  centr^  stock  leveling  system  uses  base-spe¬ 
cific  data  reflecting  past  demands  over  a  specified  period  of  observa¬ 
tion. 

A  short  time  later,  Brown  and  Geisler  [6]  examined  data  describing 
spares  demands  <m  B-50D  aircraft  frcnn  a  wing  deployed  to  RAF 
Station  Upper  Heyford,  England,  during  a  90-day  period,  and  con¬ 
trasted  them  with  demands  on  B-47  aircraft  deployed  to  Fairford, 
England,  for  a  similar  period.  Little  of  importance  was  noted  beyond 
the  larger  numbers  of  demands  per  flying  hour  experienced  by  the 
B-SOs,  especially  for  small  hardware  items,  not  surprising  in  view  of 
the  difference  in  technologies  represented  by  the  two  types  of  aircraft. 
Karr  [7]  related  the  B-47  database  describing  spares  demands  with 
data  describing  AOCP  (aircraft  out  of  commission  for  parts)  occur¬ 
rences.  He  analyzed  ei^t  weeks  of  AOCP  reports  from  two  B-^  " 
bases  (March  and  MacDill).  AOCPs  are  roughly  equivalent  to  MICAP 
occurrences  in  the  modem  Air  Force.  IBs  observations  seem 
consistent  with  the  distributions  of  parts  shortages  among  aircraft  to 
this  day,  and  serve  to  point  out  the  important  role  of  consolidating 
shortages  (cannibalizing)  among  aircraft  as  well  as  the  inhibiting  ef¬ 
fect  of  shortages  of  parts  that  cannot  readily  be  cannibalized.  His 
conclusions  included  the  following  observations: 

1.  Most  of  the  shortages  were  corrected  within  a  few  days  (91  per  cent 
appeared  in  only  one  weekly  report). 

2.  Most  of  the  out  of  commission  aircraft  lacked  only  a  single  part  dur¬ 
ing  the  eight  week  period  studied  (60  per  cent  lacked  one  part,  23  per 
cent  lacked  two  parts). 

3.  There  were  several  aircraft  in  a  hangar  queen  condition  (11  lacked  5 
or  more  parts). 

4.  A  large  proportion  of  the  aircraft  were  out  of  commission  because  of 
lack  of  parts  during  the  eq^t  week  period  studied  (71  per  cent  of  all  air¬ 
craft  on  the  two  bases  studied  were  out  of  commission  at  one  time  or 
another). 
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5.  The  dMaumd  rate*  far  th*  laAiag  ipvte  were  typically  vny  low  (52 
per  cent  had  experienced  aero  demand  during  a  1300  airen^mtmth 
sanqple). 


6.  Mortoftheladdi^  parte  were  ewyinexpenaieB  (87  par  ceirteeetleea 
than  $10). 

Our  guess  is  that,  with  an  a4iustmai.t  for  inflatkm,  Karr's  obsorva- 
tioos  would  e^Hldy  quite  well  to  the  current  world. 

With  the  realization  that  dmnands  fm:  aircraft  qtares  were  not  ade¬ 
quately  described  by  the  Poisson  distribution.  Youngs,  Geisler,  and 
Mirkovich  [8]  puUiBhed  an  BM  describing  confidence  intervals  fiir 
Poisson  parameters  in  logistics  research.  On  the  face  of  it,  this  would 
seem  to  ignore  the  obvious,  but  it  was  simply  a  first  step  in  a  move 
away  firom  the  Poisson  to  &e  negative  binomial  probability  distribu- 
ti<m.  it  was,  in  fiict,  an  explicit  recognition  of  the  fact  that  past  de¬ 
mands  may  not  accurately  represent  an  item’s  demand  rate. 

The  first  ezididt  reference  in  the  eariy  RAND  litaature  to  tiie  use  of 
negative  binomial  probability  distributions  to  describe  spares  de¬ 
mands  occurred  in  an  RM  by  Youngs,  Geider,  and  Brown  [9].  In  this 
case,  the  negative  binomial  distribution  was  used  to  describe  the  con¬ 
ditional  probability  of  obsm-ving  y  demands  in  some  specified  future 
time  poriod  having  observed  z  demands  in  some  past  pmriod.  The  ex¬ 
perience  base  was  chosen  to  be  aircraft-m<mths  of  experience.  The 
negative  binomial  model  was  based  on  the  assumption  of  a  Poisson 
demand  process  with  a  gamma-disbributed  location  parameter,  a  re¬ 
sult  published  in  1920  by  Greenwood  and  Yule  [10].  In  thmr  opening 
discussion.  Youngs,  Geider,  and  Brown  conclude  that: 

The  method  of  cimditional  probainlities  is  much  m«»e  precise  than  the 
strai^t-forward  Poisson  cq>proach  for  the  low  probabilities  and  fire- 
quonriee  of  demand,  wfaidi  is  very  important  in  the  case  of  aircraft 
qmres.  For  the  higher  demand  rates,  (demand  rates  greater  than  0.5 
per  100  aircraft  mmtths)  the  two  approaches  give  very  similar  results. 
Also,  the  results  of  the  two  methods  converge  as  the  eqimienoe  period 
gets  longer. 

The  oondusiim  about  convergence  was  based  on  the  assumption  of  a 
steadi-state  process  involving  an  unknown  constant  location  parame¬ 
ter.  It  was  imt  based  on  the  data.  Ghven  vriiat  we  have  subsequently 
observed  about  spares  demand  processes  using  large  datasets,  it  is  in¬ 
correct. 

In  February  1955,  Geisler  [11]  documented  a  briefing  he  gave  to  the 
Long  Range  Logifrtics  Research  (Conference  in  Santa  Monica.  He 
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sumoiarized  some  cf  the  analyses  oi  demand  data  RAND  had  done  up 
until  tiiat  time  and  suggested  the  desirahility  of  increasing  the  rtodc 
levels  of  low-oost  items  at  base  level.  Shortly  after  Geisler’s  briefing, 
he  and  Mirkovieh  [12]  puUished  analyses  ci  spare  parts  dmnands  on 
F-86D  aircraft  and  noted  many  similarities  with  those  on  B-47  air¬ 
craft.  The  same  authors  [13]  reported  analyses  of  worldwide  spares 
dmnand  data  that  reinforced  the  observations  muda  by  Geisler  in  his 
briefing.  A  short  time  later.  Hamburger  [14]  suggested  the  evalua¬ 
tion  of  simplified  distribution  fiinctions  to  describe  item  demand.  His 
work  apparently  received  little  attention;  however,  in  comparing  the 
simpler  fiinctions  that  he  suggested  with  the  Poisson  distribution,  he 
wrote: 

Demand  predictions  are  affixted  by  three  major  sources  of  error: 

1)  The  random  occurrence  of  demands:  Actual  dmnands  fluctuate 
about  the  true  demand  rate  in  a  iadiion  described  by  the  Poisson  distri¬ 
bution.  These  fluctuations  are  particularly  noticeable  when  periods  of 
time  which  are  shnrt,  relative  to  the  demand  rate,  are  ctmsidered  (e.g., 
the  number  ci  automdbile  accidents  occurring  over  a  weekend  can  be 
predicted  more  precisely  than  the  number  occurring  in  the  course  of  an 
hour). 

2)  Insufficient  information  for  determining  true  demand  rates:  It  may 
be  that  the  true  demand  rate  of  parts  with  an  experienced  demand  of  5 
is  actually  10^  on  the  average.  But  some  irf’  these  parts  will  have  hi^ier 
than  average  true  demand  rates,  while  others  have  lower  ones. 

3)  Biased  estimates  of  true  demand  rates:  The  estimated  average 
true  demand  rate  <d'  a  group  of  parts  may  be  incmrect. 

It  was  not  tile  first  allusion  to  the  idea  of  a  ‘Hrue  demand  rate,”  nor 
would  it  be  the  last 

An  RM  by  Karr,  Geisler,  and  Brown  [15]  was  the  first  of  several  that 
explored  the  design  of  flyaway  kits  (war  reserve  spares  kits  in  the 
modem  Air  Force).  Hamburger  [16]  also  explored  an  alternative  ap- 
proadi  to  computing  flyaway  kits.  Qark  [17]  offered  an  approach  to 
the  central  allocation  of  stock  levds  among  bases  and  the  depot  in  the 
anticipated  future  Air  Force  environment  in  which  computers  would 
be  availaUe  to  cany  out  the  tedious  computations  involved.  It  is  in¬ 
teresting  to  note  that  Clark’s  approach  drew  on  the  use  of  worldwide 
data  to  sufqxnt  such  allocations  oonsistmit  with  tiie  earlim*  observa¬ 
tion  Iqr  Geisler  and  Youngs  that  base-specific  demands  were  too 
sparse  for  such  purposes. 
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Berman  [18]  postolated  a  linear  programming  ai^noach  to  modeling 
the  procnminent-repair  dedsimi  fiv  an  item  that  tried  to  acooont  fier 
its  life-^rde  costs. 

The  notion  that  low  demand  imidies  uninedictability  pervades  this 
early  literature.  Time  and  again  sparse  demands  were  Uamed  fiar  the 
inability  to  forecast  future  demand  and  estimate  the  demand  distri- 
Imtion  adequately.  In  later  yean,  advances  in  probalnlity  theory, 
spedfieally  Bayesian  finmulations,  brou^dt^t  to  researchers  a  more 
constru^ve  view  ai  the  proUem. 

The  sequence  of  RMs  described  thus  Cbut  culminated  in  the  publication 
of  a  formal  RAND  report  ly  Bernice  ftrawn  in  July  1856  [19]. 
EntMed  Charactaistiea  of  Demand  fist  Aircraft  Spare  Parte,  it  sum¬ 
marized  the  work  that  had  been  done  until  that  time.  The  Summary 
of  Brown’s  report  is  of  special  interest  here.  Ebrcerpts  follow. 

Koovdedga  of  demand  fin*  aircraft  parts  is  needed  tat  effective  and  eco¬ 
nomical  i»ocurement,  distribution,  and  stoekage  decisions.  The  follow¬ 
ing  paragraphs  sommarize  the  results  of  RAND  research  on  Hemami 
ai^  point  to  certain  conclusions  that  can  be  ^wn  for  the  logistics  qns- 
tem. 

Low  average  demand  rates  ate  characteristic  of  a  large  proportitm  of  all 
aircraft  parts.  During  a  year's  period  at  the  bases  studi^  (and  perhaps 
at  all  bases),  as  many  as  one-third  of  the  available  spare  parts  had  no 
demand,  and  three-fourths  had  so  few  demands  that  l^y  i^r  an  uiue- 
liable  basis  for  inedicting  future  demand.  Idcneover,  many  of  the  parts 
had  low  unit  costs.  Forty-two  per  cent  of  aU  line  items  in  the  USAF 
Woridwide  Stock  Balance  and  Consumption  Report  for  1962-1953  had 
fewer  than  ten  issues  during  tiie  year  cost  less  than  $10  each. 

slow-moving,  low-cost  parts  account  for  a  small  fractim  of  the  total 
dollar  value  of  issues,  but  because  of  their  large  number  and,  often, 
thw  essentiality  to  the  ftmctioning  of  the  airoaft,  they  constitute  a 
significant  logistics  proUem. 

Demand  for  nmet  spare  parts  also  tends  to  be  erratic.  Even  if  the  de¬ 
mand  rate  for  a  part  is  l^wn  for  some  past  period,  the  fiiture  demand 
during  a  similar  period  cannot  be  predicted  with  accuracy.  To  reduce 
the  occurrence  of  parts  shortages  to  a  reasonably  low  level,  it  is  not 
enough  to  predict  (and  use)  average  demand  rates,  but,  rathm*,  it  is  nec¬ 
essary  to  predict  the  probability  that  various  demands  occur. 
These  probabilities  can  then  be  used  in  calculatums  designed  to  focUi- 
tate  stoekage,  procurement,  and  otiier  important  logistics  dedskms. 

In  many  cases,  the  pixmounoed  randmn  elemmit  eauang  uncertainty  in 
demand  can  be  ezprasaed  by  a  mathematical  fimnula.  Bfanyoftiie  air- 
firame  parts— with  sufiBdentty  firequent  demands  to  permit  statistical 
analyses— show  demand  imttems  that  can  be  approsimated  Ity  stan¬ 
dard  probability  distributions,  sudi  as  the  FoUnon  probability  distribu¬ 
tion.  When  si^  apinuzimatimi  is  possible,  logistics  demons  can  be 


rcaaputod  that  ta]»  into  account  the  coat«  of  iiynimag  ■bortagee  aa  well 
•stheeoataofamMiiigthaniL  This  can  nsolt  in  storage,  procurament, 
and  odier  logisties  dedWons  that  inodiioe  greater  coml^  capaUlity  for 
tibe  resooroes  available. 

There  are  many  eaitsee  of  the  oniaadictahility  of  demand  for  qMure 
parts.... 

The  foct  mnains  that  demand  for  most  q>are  parte  cannot  now  be  pre¬ 
dicted  vrith  oonfidnwe,  and  perhaps  never  can.  This  makes  it  neoessary 
to  consider  some  inqnrovements  in  logistics  operations  to  make  it  easier 
to  live  with  demand  uncertainty.  Among  sn^  unprovements  would  be 
a  shortening  of  the  resi^ndy  time,  of  the  procurement  lead  time,  and  of 
the  repair  orde  for  spare  parts.  Eadi  of  these  innwrovements  would 
hdp  to  reduce  die  time  over  whidi  iwedidiMis  must  be  made  and  would 
leam  requiremeiits  for  procurement,  thus  reducing  the  risk  attending 
prediction: 

1.  Shortening  resiq^y  timo  would  generally  reduce  the  amount  of 
buffor  sto^  that  must  be  kept  at  air  bases.  Ihese  buffer  stodu  are 
now  large  and  costly  because  it  is  so  difBcolt  to  predict  demand  at  base 
level.  Even  thou^  it  mi|d>^  cost  the  qntem  more  to  reduce  resuiqity 
time,  the  savingi  in  required  bidfer  sto^,  as  wdl  as  the  reduettem  in 
lost  performance  time  for  aircraft  suffering  the  shortage,  mi^  out- 
wei^  this  increase  in  cost. 

2.  Reducing  the  procurement  lead  time  would  promise  considerable 
ecanomiea  in  the  procurement  of  spare  parts.  In  the  early  stages  of 
prodnetfoo,  vdien  there  is  little  demand  etyierience  and  much  statistical 
unonrtainty,  short  i»ocorem«it  lead  times,  with  the  option  of  frequent 
reorder,  w^d  help  much  to  ecawmixe  m  procurement.  It  is  also  obvi¬ 
ous  that  demand  predictkai  at  all  stages  is  hampered  by  many  dynamic 
elwnents,  sudi  as  uneqweted  changes  in  aircraft  configuration,  in  en¬ 
gineering  design,  and  in  aircraft  procurement  schedules.  A  shortening 
^  procurement  lead  time  would  reduce  tiie  impact  of  these  elements. 
Such  shortening  might  be  hard  to  achieve  because  it  would  probabty  re¬ 
quire  changes  in  contractual  and  procurement  techniques  used  by  the 
U.S.  Air  Force.  RAND  is  domg  research  along  these  li^.  Very  likely, 
leductHms  in  procurement  lead  times  would  be  accompanied  by  fo* 
creaaee  in  unit  cost,  but  these  increases  shotild  be  more  than  counter¬ 
balanced  by  reductiais  in  the  vdume  of  parts  procured. 

3.  A  shortening  repair-cycle  time,  finally,  could  ]»obably  be  aeoom- 
idished  only  by  mqjOT  roviskms  in  the  present  system  of  scheduliity  and 
dmng  repair.  lUs  diortening  would  have  ffie  same  benefits  as  the 
shortening  of  procurement  lead  time.  In  the  early  stages  rtf' prodoctirm, 
fost  repair  wo^d  permit  the  system  to  operate  with  a  snmller  inventory 

parts;  and  at  all  stages,  it  would  cushion  the  uncertainties  of  d^ 
ma^  The  aldity  to  repair  quickty  would  require  mudi  mme  rapid 
transminkmcffdatabetweenbaaesairddepotstmdmoreimmediatete- 
actfons  by  the  depots  to  such  data.  Such  rovishms  in  the  repair  system 
would  undoubted  result  in  hitler  unit  rqwir  costs,  but  here  agedn  the 
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final  oirtwia*  wcwM  ptobdbly  b>  a  n«t  fBdMetion  in  total  cort  to  tile  ^yg- 
tem  tlmni^  laaa  prociiraogMnt  and  fiwar  ahoitagM. 

IliaaaiiBpwwgwmtamkgiatkal  manatmnant,  ifn>ada.iw>aldi»Bbably 
hava  valuable  eflbcta  in  various  directi^  that  lie  outside  the  scope  oi 
tbisstudjr.  Inrelatkmtothafi)reeariiag(d^deiBaDd,thiqrwouldtei^to 
overeome  some  of  the  costly  efbcts  of  &  veiy  limited  predictability  c( 
demand  fbr  aircraft  spare  parts. 

These  remarks  have  a  remarkable  resonance  for  those  of  us  invidved 
in  militaiy  logistics  researdi  at  RAND  today,  36  years  later.  Wehave 
esaentialty  oome  full  drde,  having  recently  rediscovered,  in  a  sense, 
the  core  issues  invtdved  in  eCEective  logistics  support  ci  military  op¬ 
erations.  In  recent  years,  our  motivation  has  been  to  identity  and 
evaluate  alternative  approaches  to  improving  the  logistics  tystem’s 
alntity  to  cope  with  uncertainty.  In  some  ways,  our  current  view  of 
the  proUem  is  undianged  firmn  that  which  emerged  from  the  worii  of 
Brown,  Geisler,  and  others  in  the  mid-1950s;  however,  the  source  of 
uncertainty  is  diffinrent,  at  least  in  part  It  derives  not  onty  firmn  the 
chfficulty  in  predicting  peacetime  demands,  but  also  firom  the  uncer¬ 
tainties  of  wartime.  As  was  pointed  out  by  Cohen,  Abell,  and  Lip^tt 
[201,  the  unpredictalniity  of  r^ouroe  danands  in  peacetime  is  bkdy 
to  be  compounded  in  vrartime  by  system  dimmptions,  resource  losses, 
mid  tile  inevitaUe  surprises  of  the  combat  scenario.  The  solution  di¬ 
rections,  however,  are  essmtially  the  same  as  those  suggested  by 
Brown:  flexibility  and  reqioiudveness  in  the  logistics  sirstem  to  help 
mitigate  our  inability  to  finrecast  resource  demands.  This  is  the  prin¬ 
cipal  thrust  of  our  work  in  spares  and  repair  requirements  estima¬ 
tion.  Nevertheless,  we  have  explored,  and  omtinue  to  ejqplme,  our 
ability  to  improve  the  modeling  of  peacetime  demands  for  aircraft 
spare  parts,  a  problem  to  which  we  are  bringing  a  collection  of  meth¬ 
ods  and  ideas  that  were  not  well  known  at  the  time  of  the  earlier 
RAND  researdi.  Thus,  as  remarkably  insi^tful  as  the  Brown 
Summary  is,  much  has  been  learned  in  the  logistics  research  com¬ 
munity  in  t^  intervening  years  that  now  enables  us  to  add  to  the 
bod^  of  knowledge  that  emerged  finm  this  early  woik,  althou^  the 
principal  thrusts  of  the  conclusions  in  tiie  Brown  Summary  seem  to 
be  as  ai^HTopriate  today  as  they  were  then. 

Remarks  found  throu^out  the  early  puUications  reflect  imjtiicit  as¬ 
sumptions  ribout  tile  uncertainty  in  demands  for  aircraft  spare  parts 
that,  ahhon^  important  to  any  effiwt  to  understand  them,  are  not 
mate  eqilkitiy  dear.  Those  assumptions  are: 


•  IteaaM  have  troe  demand  ratae  whidi  are  unknown  OHigtanta.  Ifwe 
had  enou^  data,  we  could  estimate  them  better. 

•  Soimd  dedskiia  about  stocfaige  of  items  with  no  or  low  observed 
demand  are  intraetabte  because  we  are  unaUe  to  estimate  proba¬ 
bility  distribtttUnis  to  describe  their  future  demands;  therrfme,  we 
are  Kkdy  to  relatively  larger  errors  in  predicting  demands 
for  these  itons. 

•  Items  with  large  numbors  demands  in  past  time  periods  are 
more  tractable  because  we  are  aUe  to  estimate  the  probability  dis¬ 
tributions  of  their  future  demands,  but  we  still  have  to  ctmtend 
with  the  proUem  that  we  don’t  really  know  their  true  demand 


As  we  will  show  in  the  sections  that  follow,  eadi  of  these  several  fon- 
damental  amumptkms  is  incorrect  There  is  a  simple  observation  to 
makw  about  the  latter  two  of  these  assumptions.  It  has  to  do  with 
rdative  error  as  opposed  to  abecdute  error.  If  I  predict  one  event  and 
two  occur,  I  will  be  wrong  by  100  percent  If  I  imdict  100  events  and 
150  occur,  I  wdl  be  wrong  fay  only  50  percent  The  outcome  in  the 
second  case  hardly  makes  me  a  befctmr  forecaster.  If  every  event  that 
contnfautes  to  my  forecasting  error  costs  me  $1,000, 1  would  hate  to 
be  faced  with  the  latter  outcome.  Similarly,  if  we  have  larger  relative 
error  in  forecasting  demands  for  low-demand  items,  it  won’t  neces¬ 
sarily  have  serious  adverse  effect  on  the  performance  of  the  invaatory 
systra.  In  foct,  fairly  modest  rdative  errors  in  forecasting  demands 
for  hi^-demand,  hicd^-cost  items  can  be  for  more  costly  either  in  per¬ 
formance  degradation  or  unwise  investments. 

An  important  motivating  factor  for  the  interest  in  low-demand  items 
may  have  derived  from  the  study  by  Karr  [7]  in  which  he  analyzed 
aircraft  AOCP  data  frtmi  March  and  MacDiU  AFBs  from  9  April  to  4 
June  1954.  The  B-47  demand  data  described  above  were  also  used  in 
tliiii  analysis.  That  database  covered  a  different  time  period  at  each 
of  three  bases;  each  time  period  ended  in  1953.  He  found  that  50  per- 
omt  the  AOCP  occurrences  in  the  ei^t-week  period  he  examined 
were  fm*  parts  that  had  no  observed  demand  in  the  1,300  aircraft- 
month  database.  An  additional  11  percent  had  only  one  demand  in 
the  eather  data.  Karr  oonduded  tfaiat.  The  reascm  wl^  tihrae  parts 
predominate  in  spite  id  their  low  demand  rates  is  that  there  are  so 
many  of  tiwm.”  Indeed,  the  distributicm  worldwide  demands  over 
stock  numbers  today  ediibits  similar  charactoristics,  at  least  for  re¬ 
coverable  items. 


Anothor  obacrvatkm  made  in  the  eaiiy  days  deserves  oqiticit  mentum 
here:  *We  oould  find  no  significant  ooRslatioa  betwem  the  kinds  er 
quantity  of  items  deamnded  and  the  faring  activity  ci  the  airorait, 
measured  either  in  flyii^  hours,  landings  or  aircraft  mcmths.”  [3,  p.  ii] 
In  bet,  the  work  done  hefive  ftwwn’s  report  used  aircraft  m<mths  as 
the  denominator  tlie  demand  rate. 

There  are  several  important  observations  to  be  made  about  this  early 
work: 

•  All  of  the  anatyses  published,  with  the  sin^e  exception  of  that  of 
issue  data  firem  the  USAF  Worldwide  Stock  Balance  and 
Consumption  Bepmt  by  Geisler  and  Bfirkovidi  [13],  focused  tm 
base-level  demandtdata  rather  than  systemwide  data. 

•  All  of  the  published  woik,  without  exception,  pooled  data  on  con¬ 
sumables  and  reooverables  and  did  not  expHcitty  comment  on  dif¬ 
ferences  between  them. 

•  No  explicit  ocmsideration  was  given  to  interchangeability  and  sub> 
stitutability  relationships  among  line  items  in  the  inventoiy  sys¬ 
tem. 

Later  work  by  Goldman  [21]  pelted  out  several  errors  in  the  ap- 
proadies  talmn  until  that  time.  His  three  principal  condusioas,  al- 
thou^  obvious  to  any  researcher  in  the  field  now,  illuminated  the 
problem  considerably  at  the  time: 

1.  The  fomily  of  parts  rather  than  the  individual  part  number  should 
be  the  baak  unit  in  demand  anabw  and  forecasting. 

Use  id  the  family  of  parts,  consisting  of  the  master  part  number  and  all 
subsidiary  part  numbers,  as  the  bo^  unit  of  analysis  makes  it  possible 
to  take  sub^tution  relationships  tantmg  line  items  into  account  in  in¬ 
terpreting  consumption  data. . . . 

2.  When  the  data  are  anahmd  in  accordance  with  the  foregoing  consid¬ 
erations,  foture  demands  for  spare  parts  can  be  predicted  from  pro- 

'  gram-element  data. 

Goldman  [22]  subsequently  conducted  an  eqieriment  involving  the 
tradftiimal  initial  provisioning  problem  of  predicting  demand  rates  fm 
parts  without  previous  consumption  experience.  Ife  later  extended 
his  work  [23]  to  a  priori  demand  prediction  for  F-lOO  aiiftame  parts 
during  the  acquisition  phase  that  pn^[ram. 

The  problmn  of  initial  provisioning  and  support  of  weapim  systems  in 
the  early  stages  of  their  life  cycles  was  a  topic  that  absorbed  consider- 
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able  attaptiop  firam  RAND  rmaarchw  in  tiie  eariy  yean.  Some  of 
tile  fagareacea  about  parta  demand  in  the  aaify  worii  fiaind  their  way 
into  tile  IWhm  BMdd  deaeribed  by  Uodotiilin  et  aL  [24],  and  need  in 
Labovatoey  PRQeet  I  (IP  I).  IP  I  was  a  ahnulation  alterna- 

tim  legMto  8ii|9ort  pdiciea  and  atrategiea  whoae  simulated  perior- 
manoe  was  to  be  compared  with  tiie  sopport  policies  strategies 
traditionally  employed  in  the  aoqaistti<m  phase  of  a  wei^Km  system’s 
life.  Logistics  suppmt  in  the  weapon  system  acquisition  phase  pro¬ 
vided  mudi  of  the  context  for  this  early  wcnrk;  thus  the  concern  of 
these  reaearehcn  was  not  sim|dy  with  the  statistical  uncertainty  as¬ 
sociated  with  the  wwld  of  reideniahment  qiares  estimaticm  but  with  a 
worid  in  udiich  data  are  sparse  and  the  difficulty  of  the  estimation 
proUmn  thmdiiy  compoumfed. 

• 

The  initial  |»oviaioning  proUem  provided  the  ccmtezt  for  an  RM  by 
McCRothlin  and  Radnor  [26]  whidi  suggested  the  use  of  Bayesian 
tedmiques  for  pooling  early  observations  of  demand  with  initiiil  esti¬ 
mates  of  demand  rates  in  a  systematic  way  to  give  the  proper  wei^t 
to  observed  data  in  revising  the  initial  estimates  in  the  early  life  of  a 
weapon  system.  In  a  subsequent  RM,  McGlothlin  and  Bean  [26]  sug¬ 
gested  procedures  for  implementing  the  Bayesian  approach. 

An  RM  by  Astrachan,  Brown,  and  Hou|difen  [27]  reported  mixed  re¬ 
sults  from  the  aiqdicatioa  seven  different  predictive  techniques  to 
Fakon  missile  and  B-52  parta  No  particular  tedmique  emerged  as 
dearly  supmior  to  the  others. 

On  25  and  26  January  1962,  RAND  qionsored  a  Demand  Prediction 
Cknference  whidi  was  hdd  at  Stanford  University.  It  was  attended 
by  several  distinguished  academicians  and  resea^ers  who  were  or 
had  bemi  involved  with  inroUems  in  demand  prediction.  Kenneth  J. 
Arrow  contributed  a  paper  after  the  conference  which  was  published 
in  an  RM  edited  by  Astrachan  and  Cahn  [28].  Arrow’s  paper  sum¬ 
marized  the  discussions  that  to<fe  place  during  the  conference  and  in¬ 
cluded  some  observations  his  own.  His  observations  seem  to  be  es- 
pedaUy  pertinent  and  indsive  in  the  omtezt  of  the  present  work. 
Arrow  commented: 

It  is  md  easy  to  form  an  a  priori  <qrinion  about  the  fruitfulness  of  statis¬ 
tical  forseaming  tedmkioss.  This  needs  to  be  done  empiria^.  In  me 
way  or  another,  most  d  theae  methods,  apart  from  spectral  analysis, 
seem  to  be  of  the  discounted  least-squares  type. 

For  what  modd  at  the  world  would  this  statistical  method  be  correct  in 
aiqr  sense?  When  we  consider  that  we  are  getting  a  set  of  observations 
firm  virtually  the  same  universe,  why  is  evwy  obeervatim  not  as  good 
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as  may  othar  ooa?  It  ia  net  clear  iriiy  one  aho^  w«i^  tlMm  in  any 
aray. 

The  interpretation  made  by  Winters  was  that  we  can  think  of  the  pa- 
raaaetera  aa  diifting;  but  shifting  in  a  random-walk  manner,  adikii 
adds  one  more  unknown.  Looking  ahead,  tbingi  are  getting  more  and 
mere  nnoartain,  and  diaeoiinting  coa^eosatas  ftr  tine  growing  unrelia¬ 
bility.  By  the  same  token,  ifwe  start  firom  the  present  the  peat  data  are 
more  uncertain.  Thus,  if  we  arrived  where  we  are  now  I7  a  random- 
walk  process,  we  can  also  go  baekwarda  by  the  same  process.  Thismi^ 
be  the  rationalisation  for  this  kind  of  leaatwqoaree  method.  We  have  a 
past  marked  by  chang^  we  think  change  will  peraiat  aa  we  go  furtiier 
into  the  future.  The  diaeounted  least-equares  meUiode  compensate  for 
this  process  in  some  way. 

Brown's  arguments  have  shown  the  flexibility  of  this  model.  One  can 
Intild  a  great  deal  into  it,  iqqiarently,  much  more  than  by  strait  expo¬ 
nential  smoothing.  Futthumore,  it  is  possible  to  bring  in  any  eiqilana- 
twy  variaUee  we  Hke,  sodi  as  program  dements  and  age  of  parts.  A 
combination  of  smoothing  techniques  may  i»oduoe  better  results  with 
different  program  elements. 

Another  pdnt  raised  oonoemed  the  program  elements  themselves. 
Assuming  that  usage  does  have  established  relationships  to  some  pro¬ 
gram  dements,  then  in  order  to  forecast  usage  we  also  have  to  forecast 
the  program  elements.  Doing  so  introduces  ^dititmd  “noise.’’  Brown's 
argummit  is  that  it  is  hatter  to  use,  as  explanatory  variables,  mathe¬ 
matical  ftinctions  of  time  about  whoae  extrapdation  there  is  no  ques¬ 
tion. 

There  is  a  counterargument  aduch  depend  on  the  use  you  can  make  of 
the  forecast.  A  conditiond  forecast  gives  some  infinrmatimi  that  an  un¬ 
conditional  forecast  does  not  It  tdls  08  what  wiU  happen  if  we  diange 
our  minds,  so  that  we  mi^xt  say,  after  hwkiDg  at  it  “It's  reaUy  too  ex¬ 
pensive  to  fly  those  things  arou^  Td  bdter  not  do  it*  This  would 
make  our  forecasting  worse,  but  it  would  also  answer  a  question  that 
could  not  be  answered  with  an  onccmditional  forecast. . . . 

Arrow's  comments,  like  Goldman’s  earlier  observations,  seem  obvious, 
but  <mly  in  retrospect  What  we  know  the  most  about  is  what  is  hap¬ 
pening  now.  Data  firom  the  distant  past  may  be  less  pertinent  than 
data  we  have  frmn  the  recent  past;  similarly,  the  longer  our  planning 
horizon,  the  less  reliable  our  sense  of  the  future,  and  the  broader  our 
confidence  intervals  need  to  be.  Moreover,  we  may  discover  an  un¬ 
conditional  forecasting  m^od  (i.e.,  one  that  is  not  related  to  program 
values  sudi  as  ftying  hours)  wlmse  pmrformance  dominates  ^  (fthers 
by  some  error  measure,  but  if  the  world  is  changing,  we  are  more 
likefy  to  make  sensible  forecasts  with  a  conditional  method  evmi  at 
the  expense  of  larger  expected  errors. 
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Campbdl  [29]  cMiniited  denumd  data  from  the  Air  Force's  mainte¬ 
nance  data-ooUection  system  for  a  squadron  of  B-52  aircraft  over  a 
fiair-month  period  using  multi|de  corrdation  and  r^ression  analysis 
to  ezfdore  relationships  between  demands  and  seven  operational 
variables:  sorties  flown,  flying  hmirs,  flying  hours  at  low  altitude, 
bomlnng-navigation  training  units,  fins  control  system  usage,  ECM 
system  usage,  and  periodic  inspections.  He  conduded  that  demands 
seemed  to  be  related  to  flying  hours  and  sorties,  with  flying  hours 
having  the  stronger  relationship.  Campbell’s  sample  induded  recov- 
eraUe  items  mly,  not  consumaUes,  apparently  the  first  such  distinc¬ 
tion  made  in  the  earlier  RAND  research.  He  made  an  important  ob¬ 
servation  in  his  dosing  remarks: 

■nie  sharply  declining  predictability  of  component  demands  at  lower 
levels  of  aggregation  soggmts  a  concluding  thouid^t.  Some  infinmation 
is  contained  in  the  prediction  by  mqjor  system  aggregate  or  shop  aggre¬ 
gate  that  is  lost  when  only  separate  pr^ctions  are  made.  Aggregate 
predictions  can  never  replace  line-item  predictions  for  all  support  Unc¬ 
tions,  but  we  must  find  wider  applications  for  them. 

Feeney,  Petersen,  and  Sheihrooke  [30]  described  the  evaluation,  us¬ 
ing  actual  base  demand  data  for  recoverable  items  observed  at 
Andrews  AFB,  of  a  base  stodmge  polity  that  incorporated  demand 
rates  estimated  with  Bayesian  procediures  that  seemed  to  respond  di¬ 
rectly  to  Campbell’s  ideas  for  aggr^ating  data  across  items.  Stock 
levels  were  computed  using  six  months  of  past  demands  and  were 
then  evaluated  using  demands  observed  during  the  subsequent  six 
months.  The  aggr^te  fill  rate  of  the  computed  stock  levds  was 
hitler  than  that  of  both  authorized  and  on-hand  stock  levels,  and  dif¬ 
fered  firom  predicted  perfimnance  by  less  than  5  percent,  a  d^erenoe 
that  declined  with  postulated  investment  level.  It  was  a  landmark 
study  in  the  sense  that  it  combined  the  results  of  demand  prediction 
work  done  up  to  that  time  with  a  Bayesian  approach  to  the  determi¬ 
nation  of  stock  levels  that  pooled  information  across  line  items,  a  pro¬ 
cedure  that  had  previously  been  done  using  a  line-item-by-line-item 
apinroach.  They  also  pointed  out  the  sensitivity  of  system  perfor¬ 
mance  to  variability  in  item  demand.  The  authors  made  an  important 
observation  about  demand  variability  in  their  closing  remarks: 

Because  aiqr  storage  policy  must  operate  implicitly  or  explicitly  with 
SCUM  assumptkm  cf  dianand  varialality,  it  is  important  to  note  how  sen¬ 
sitive  stock  requirements  are  in  this  area.  Unfortunately,  there  is  a 
lack  of  data  with  which  to  estimate  base  demand  variability,  and  more 
important,  we  have  little  understanding  of  what  causes  su^  extreme 
fluctuations  in  demand.  Perhaps  a  large  part  of  variability  is  simply  er- 
nmeous  reputing.  Ifso,  improved  reporting  quality  could  produce  large 
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ndnetaoiM  in  sboik  invMtment  Cleai^,  we  need  to  know  mote  about 
the  nature  and  eauaee  ai  flaetuatune  in  demand.  Without  euch  info- 
matkm,  we  are  not  in  a  poeition  to  decide  what  portion  tt  demand  vari- 
abiUfy  ahould  legitinmtdy  be  covered  bj  baae  stodmge  policy.  It  ie 
hardy  neeeaaaiy  to  point  out  that  tiiie  ia  a  baek  queetion,  whidi  firam  a 
manacement  point  of  view  ia  aa  important  aa  dmoeing  a  qwcifie  target 
baae  ffll  rate. 

Aa  noted  in  previooa  atudiea,  more  reapcmaive  reaupply  doea  reduce  the 
amount  of  atoek  required  to  achieve  a  givmi  fill  rate.  From  thia  it  fel- 
kwa  that  in  matanrea  where  there  ia  a  aignificant  diflbrenoe  between 
baae  repair  cyde  Imigth  and  depot  reavqqdy  time,  it  would  be  ben^dal 
to  eataUiah  atoek  levda  for  an  item  aa  a  functiim  of  percentage  isaues 
baae  repaired. 

No  atockage  policy  can  eliminate  atockouta;  eiqiedited  deliveriea  will 
atill  be  required  firam  time  to  time  due  to  the  vagariea  of  damanH  But 
the  aehknmment  of  a  q>eeified  target  baae  fill  rate  with  minimum  atoek 
inveatment  will  cauae  the  coata  of  expediting  to  be  incurred  in  reaupidy- 
ing  the  hii^unit-coat  itema,  which  repreaent  a  more  eCBcient  uae  of 
aupport  reaourcea. 

Aa  we  win  see,  thia  RM  was  a  haihinger  of  the  important  body  of 
work  that  would  ultimately  emerge  from  the  collaboration  of  Feeney 
and  Sherbrooke. 

McGlothlin  [31]  refined  and  simplified  his  earfi^  Bayesian  approach 
to  weii^ting  initial  estimates  of  demand  with  observed  demand  early 
in  the  operaticmal  life  of  a  weapon  ssmtem. 

Fishman  [32]  described  the  application  of  spectral  analysis  to  base- 
level  demand  data  and  its  usefulness  in  separating  trends  and  qrclical 
effects  firam  random  events  but  the  method  never  found  its  way  into 
Air  Force  use.  Astrachan  and  Shmbrooke  [33]  evaluated  the  use  of 
exponential  smoothing  in  forecasting  demand  and  concluded  that  it 
“  .  .  .  does  not  appear  to  be  a  significantly  better  predictor  than  the 
cumulative  issue  rate  techniques  currency  being  used.”  Their  as¬ 
sessment  was  based  on  datas^  with  particular  characteristics  that 
may  have  contributed  to  this  outcome.  As  we  will  show  in  this  report, 
the  simplest  version  of  the  Kalman  filter  is  essentially  exponential 
smoothing,  and  we  have  found  it  very  effective  in  rediidng  the  mean 
absolute  deviation  in  demand  forecasting.  The  conclusion  readied  by 
Astrachan  and  Sherbrooke  may  also  have  been  due  in  part  to  the 
lengths  of  the  time  periods  involved  and  the  didoes  of  wei^ting  fac¬ 
tors. 

Feeney  and  Sherbrooke  [34]  described  the  application  of  Bayesian  in¬ 
ference  to  the  analysis  cf  spare  parts  dema^  using  a  different  ap¬ 
proach  firam  that  of  McGfiothlin.  Where  MoOlothlin  was  inferring  an 


ertiourte  of  tlM»  deoiaiid  rate  amrigning  aniropriate  wog^ts  to  initial 
eetimates  and  obeo-ved  data  for  a  dnide  item,  Feen^  and  Sherl»o<die 
■oggeeted  an  amaofldbi  that  eetiinated  an  item’a  demand  rate  by  (rf>- 
serving  the  durtributkma  oi  demand  rates  of  all  items,  lliey  argued 
that  sohetantial  impravements  could  be  made  by  mring  qrstem* 
oriented  approaches  to  su|q)ly  management  and  demand  modeling 
over  traditumal  item*orimited  approaches  and  demonstrated 
elementary  examples  of  this  thinking.  The  aiqproach  th^  d^cribed 
has  fundamentally  important  implications  for  the  demand  modding 
proUon.  They  make  the  fidlowing  observations  in  their  summary: 

The  traditioaal  qiproadi  to  demand  analysia  calculates  an  item's  issue 
rate  (demand  observed  over  scane  past  period  divided  1^  the  length  of 
the  period)  and  rivoumes  that  future  demand  will  be  some  random  vari- 
atkm  aroimd  tl.  s  levd.  Sudi  an  approach  is  adequate  if^  item  has 
relatively  demand,  if  a  relatively  kmg  bistcny  is  available,  and  if 
the  hist^  is  relevant  But  many  of  ^  most  impcartant  items,  particu¬ 
larly  highcost,  low-demand  spare  parts,  fidl  to  meet  one  or  more  of 
thoM  three  essimtittl  requirements. 

A  new  iq^aooch  to  demand  analysis,  based  on  a  mathematical  tech¬ 
nique  calMBegwsianui/fereiio^  is  described  in  this  Memorandum.  This 
approach  expk^  the  suriniaing  &ct  that  we  can  increase  our  knowl- 
e^  of  an  item  by  analyst  the  behavior  of  the  other  items  in  the  sys¬ 
tem.  Instead  ci  tiying  to  estimate  the  itmn’s  true  average  demand,  this 
approadi  estimates  the  probalnlity  that  the  item’s  average  demand  is  at 
one  of  several  levels.  We  think  this  is  a  much  more  realistic  way  of 
characterizing  what  we  can  leam  firum  an  item’s  demand  histmy. 
These  probabilities  can  now  be  used  to  appraise  the  true  risks  and  po¬ 
tential  payoff  fin*  various  stock  levels.  I^ta-processing  procedures  can 
be  designed  to  use  infimnation  ci  this  kind  vrith  littie  or  no  increase  in 
processing  complezi^. 

Bayesian  demand  analysis  has  two  immediate  implications  for  im¬ 
proved  snpidy  management.  First,  because  it  wrings  irmTinnim  rele¬ 
vant  infinmation  from  available  demand  data,  this  approach  promises 
subrtantial  improvement  in  the  efflkaenQr  of  stodmge  decisions. 
Specifically,  by  using  tins  kind  of  analysis  we  should  be  less  liai>le  to 
overestimate  ^mand  and  boy  too  much  because  of  a  random  surge  in 
demand,  and  less  liable  to  underestimate  demand  and  buy  too  little  be¬ 
cause  at  a  random  decline  in  demand.  Second,  because  the  approach 
can  be  applied  to  any  time  period,  it  is  extremely  flexible.  It  steuld  be 
possible  to  eliminate  many  of  the  policy  problems  now  created  by  items 
that  do  not  meet  the  requirements  of  daily-iasoe-rate  computatimi: 
items  with  low  demand,  and  items  with  erratic  Hanmnd  patterns.  In 
the  fiamework  of  Bayesian  demand  analsrsis,  policies  can  be  developed 
that  prescribe  action  unamlnguously. 

Feen^  and  Sherinrooke  [35]  extended  the  well  known  queuing  result 
of  Palm  [36]  to  the  proUmn  of  determining  stock  levels  for  recoverable 
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spares  uador  the  assumptum  of  compound  Pmssoa  demand,  a  mudi 
ridier  case  tlian  was  jHrevmosly  tractaUe,  althoa(^  not  ime  (m  espe¬ 
cially  s(did  logical  ground,  as  we  shall  see  later.^  Feeney  and 
Sherbrooke  [37]  further  diaeassed  the  iqiplicatiim  of  Bayedan  infer¬ 
ence  to  the  demand  estimatimi  problam  and  suggested  a  method  f<nr 
coupling  it  with  exponential  smoothing  vdiere  the  dmnand  rate  is 
ncmdationaiy.  Iliey  subeequenily  discussed  an  approach  [38]  to  set¬ 
ting  base  sto^  levels  for  recoverable  spares  that  integrated  the  woiii 
previously  described  in  Refs.  35  and  37. 

Fiess  [39]  developed  empirical  Bayes  estimators  for  the  univariate 
and  multivariate  exponential  family  of  distributions,  for  distributions 
with  nuisance  parameters,  and  for  the  distribution  of  a  family  of  ran¬ 
dom  variables  (Poisson  process),  extending  the  work  of  Robbins  [40], 
and  discussed  their  application  to  demand  modeling.  Campbell,  Lu, 
and  Michels  [41]  extmided  the  approach  of  Feenoy  and  Sherbrooke  to 
the  computation  of  war  reserve  spares  requirements  in  the  context  of 
the  aircraft  dispmrsal  pohc^  then  in  effect  in  the  Air  Defense 
Command.  Hou^ten  [^]  explored  alternative  procedures  fm  ensur¬ 
ing  that  AFLC’s  estimations  of  requirements  for  recoverable  spares 
were  consistent  with  base  stockage  policies. 

The  early  body  of  work  at  RAND  exploring  the  demand  modeling 
problem  culminated  in  the  work  of  Feeney  and  Sherbrooke.  They 
dealt  effectively  with  virtually  every  demand  modeling  problem 
raised  by  those  whose  work  preceded  theirs.  Moreover,  their  interests 
extended  beyond  the  demand  modeling  problem;  they  also  made  im¬ 
portant  contributions  to  supply  stockage  policy,  at  lecut  for  peacetime 
operating  stocks.  Feeney  loft  RAND  in  1966,  but  Sherbrooke  contin¬ 
ued  to  pursue  important  issues  in  stockage  polity.  In  a  1966  RM  [43], 
he  discussed  compound  Poisson  processes  and  their  application  to  the 
demand  modeling  pr(d>lem  in  anticipation  of  the  need  for  a  more 
tractable  form  of  probatnlity  distribution  for  use  in  a  multi-echelon 
in/entoiy  model  called  METRIC  (Multi-Echelon  Technique  for 
Recoverable  Item  C!ontrol),  which  would  have  an  important  effect  on 
inventory  management  policies  and  practices  throu^out  the  world, 
especially  in  the  large  inventory  systems  of  mihtaiy  organizations. 


^A  oompowid  Pmtaon  proctm  is  a  stochastic  process  in  whidi  the  number  o(  arrivak 
that  occur  in  a  specified  time  period  is  described  by  the  Poisson  distribution,  but  with 
eadi  arrival  more  than  one  event  may  occur.  Hie  number  of  events  that  occur  with 
ea^  arrival  is  deaerfiied  by  s  cmapouading  distribution.  In  the  {wesent  context,  an 
arrival  would  be  represented  by  file  receipt  rfa  requisition  fi>r  spare  parts;  the  quantity 
of  parts  requested  on  each  requisitioa  would  be  the  oompounding  random  variable. 
Thus  the  distribution  of  the  number  of  parts  requested  in  tiie  time  period  would  have  a 
hi^mr  variance  than  the  simple  Poisson  arrival  process. 


METRIC  [44]  extended  the  earlier  woi^  by  Feeney  and  Sherbrooke  to 
the  multi-edielon  base-depot  inventory  ^y^m.  It  assumed  no  lateral 
resupply  and  did  not  provide  for  condemnations.  Its  key  to  sdution  of 
the  multi-echelon  sy^m  was  the  application  of  Little’s  theorem  [45] 
to  the  problem  of  estimating  depot  delay  time,  i.e.,  the  time  from  re¬ 
ceipt  of  a  requisition  the  depot  to  the  shipment  of  the  item,  which 
of  course  is  a  function  of  the  depot  stock  level.  METRIC  was  de¬ 
scribed  in  laymmi’s  terms  in  a  subsequent  publication  [46].  The  views 
of  Feeney  and  Sherbrooke  also  shap^  the  aggregate  stockage  policy 
for  EOQ  items  at  base  level  suggested  by  Lu  and  Brooks  [47]  and  pub¬ 
lished  in  Jime  1968.  The  publication  of  the  METRIC  development 
was  also  followed  in  June  1969  by  an  RM  by  Fishman,  who  suggested 
alternative  methods  of  forecasting  requirements  for  depot  mainte¬ 
nance  [48],  in  which,  incidentally,  he  suggested  the  use  of  exponential 
smoothing  in  forecasting  NRTS  actions  and  clearly  demonstrated  its 
superiorily  to  an  eight-quarter  moving  average. 

METRIC  was  eventually  implemented  by  AFLC  as  the  variable  safety 
level  (VSL)  algorithm  in  1975.  It  survived  as  the  principal  ingredient 
of  safety  stock  computations  tmtil  very  recently  when  it  was  replaced 
by  the  Aircraft  Avsdlability  Model  (AAM),  a  METRlC-based  computa¬ 
tion  that  took  explicit  accovmt  of  weapon  system  complexity  in  terms 
of  the  ntunber  of  LRUs  whose  availability  affected  the  availability  of 
aircraft.  Unfortunately,  neither  of  these  implementations  took  ad¬ 
vantage  of  much  of  the  Imowledge  that  emerged  from  RAND’s  work  in 
demand  modeling.  The  mean  demand  rate  was  estimated  with  an 
ei^t-quarter  moving  average  of  worldwide  demand  counts.  More  will 
be  said  about  the  logical  flaws  of  these  procedtires  below.  In  short, 
METRIC  found  its  way  into  the  Air  Force  requirements  system,  but 
improved  techniques  for  forecasting  demand  did  not.  Despite  the 
problems  associated  with  METKIC’s  implementation,  it  was  the  sem¬ 
inal  work  in  multi-echelon  inventory  theory,  and  it  has  had  a  pro¬ 
found  influence  on  the  course  of  inventory  theoretic  developments 
ever  since. 

THE  LATER  YEARS:  1969-1990 

The  work  of  Feeney  and  Sherbrooke  was  sufficiently  satisfying,  given 
RAND’s  views  and  interests  at  the  time,  that  the  topic  of  demand 
modeling  was  not  reopened  for  more  than  a  decade.  The  occasion  for 
the  renewed  interest  in  demand  modeling  in  the  early  1980s  was  a 
shift  in  focus  fitun  the  modeling  of  peacetime  demands  for  aircraft 
spare  parts  to  logistics  support  in  wartime. 
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Amoog  ihe  iraues  that  quickly  emerged  in  the  decade  of  the  19608 
was  that  of  the  pertuihation  in  dnnand  lamxsses  induced  hy  ^ 
trantrition  from  peacetime  to  wartime.  Demands  for  resources  were 
assumed  to  be  directly  proportional  to  flaying  hours  (as  they  still  are 
by  the  current  system),  and  fljdng  hours  were  anticipated  to  increase 
(dramatically  in  wartime,  especially  in  the  early  days  of  a  war.  This 
topic  of  nonstationarity  in  demand  received  consideralde  attrition 
firom  SAND  researdiers,  and  led  to  several  significant  developments 
in  capability  assesament  methodolc^  and  some  advances  in  demand 
modiding.  In  the  first  formal  RAND  publication  dealing  with  nonsta- 
tionariiy  in  demand  in  1980,  Muckstadt  [49]  developed  an  approxi¬ 
mation  to  the  probabiliiy  distribution  of  the  number  of  items  in  re¬ 
supply  based  on  the  assumption  that  the  demand  process  was 
described  by  a  nonstationary  Poisson  distribution  and  compared  it  to 
an  exact  derivation.  The  approximation  was  shown  to  be  excellent. 
He  also  demonstrated  that,  in  the  face  of  nonstationarity  in  demand, 
it  is  important  to  model  the  nonstationarity  explicitly,  since 
assumptitms  of  stationarity  can  lead  to  serious  errors  in  the  allocation 
of  assets.  He  also  discuss^  how  the  development  could  be  applied  to 
the  initial  provisioning  problem. 

Nonstationarity  in  demand  was  an  important  feature  incorporated 
into  a  series  of  capability  assessment  models  generically  referred  to 
as  Dyna-METRIC.  In  its  original  development  by  Hillestad  [50], 
Dyna-METRIC  was  a  noteworthy  departure  firom  the  traditional 
steady-state  models  of  the  time,  ^though  the  initial  focus  of  RAND’s 
research  in  wartime  logistics  support  in  the  19808  was  on  the  transi¬ 
tion  firom  peacetime  to  wartime,  the  important  advantage  provided  by 
models  that  explicitly  accounted  for  nonstationarity  was  quickly  put 
to  good  use  in  logistics  policy  analysis  concerned  with  the  more  gen¬ 
eral  problem  of  combat  logistic  support.  A  later  version  of  Dyna- 
METRIC  [51]  was  adopted  hy  AFLC  as  its  standard  capabilily  as¬ 
sessment  model  and  incorporated  in  its  Weapon  System  Management 
Infmmation  System  (WSMIS).  Pyles  [52]  suggested  approaches  to  its 
use  in  practical  logistics  management  applications. 

These  early  versions  of  Dyna-METRIC  were  analytic  models  that 
were  relatively  efifident  in  running  time  but  lacked  some  of  the  rich¬ 
ness  needed  for  evolving  logistics  policy  studies  at  RAND.  A  later 
version  called  Dyna-METRIC  Version  5  [53]  was  a  h3d>rid  analytic- 
simulation  model  in  which  constrained  repair  capadly  and  manage¬ 
ment  adiq>tation8  such  as  cannibalization,  lateral  repair,  and  lateral 
supply  could  be  explidtly  represented.  Unfortunately,  Versitm  5  was 
not  {d)le  to  deal  with  the  indenture  relationships  among  LRUs 
and  SRUs.  A  new  version.  Version  6,  currently  being  used,  corrects 
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that  d^eieiicy.  All  of  those  develoimMmto  in  ca^iability  aooMsment 
mothodciegy  wMo  undertaken  to  pursue  polky  ioaueo  in  logioticB  re- 
aeardi  at  RAND.  Vernon  4  was  eufficittitiy  tractable  and  useful  that 
it  fimod  its  way  into  AFIXTs  standard  systems.  Later  versions  may 
also  be  imfdemented  in  the  future. 

Some  explorations  of  the  mathematiGS  associated  with  n<mstati(mazy 
demand  prooeaaes  were  d<me  more  or  less  in  parallel  with  the  devel- 
ojanent  of  these  several  versions  of  Dyna-METRIC.  Crawford  [54] 
and  Carrillo  [56]  published  rraults  tiiat  extended  Palm’s  tiieorem  [36] 
to  nonstationary  Phisson  processes  and  to  nonstationaiy  compoimd 
Pnsson  processes,  respectively.  Crawford  asserted  that  his  result  ex¬ 
tended  to  &e  compound  Poisron  case  but  did  not  treat  thi«  more  gen¬ 
eral  problem  explicitly.  In  an  explanation  for  his  approadi,  he  made 
an  observation  that  is  especially  pertinent  in  the  context  of  the  cur¬ 
rent  work: 

The  practke  of  aasuming  an  arrival  process  is  compound  Poisson  when 
the  data  exhibit  a  variance-to-mean  ratio  greater  ti^n  1,  as  is  often  ad¬ 
vocated  in  the  literature,  has  little  legitimate  justification.  If  the  vari- 
anoe-to-mean  ratio  is  aignificanily  larger  than  1  (whatever  that  means), 
the  arrival  process  is  more  often  nonhomogeneous  than  [it]  is  compound 

PoisaoD - In  inventory  applications,  treating  the  arrival  process  as  a 

compound  Pmsson  when  it  is  in  foct  nonhomogeneous  Poisson  can  be 
shown  to  have  a  significant  effect  on  the  optimal  stocking  plan. . . .  ^ 

Crawford  attributes  the  latter  observation  to  J.  Y.  Lu,  a  former  RAND 
researcher.  It  was  never  published.  As  we  will  show  in  discussion 
below,  compound  Poisson  distributions  are  applicable  to  only  a  small 
proportion  of  the  items  in  the  recoverable  inventory  management  qrs- 
tem  (the  vast  minority  of  requisitions  being  for  quantities  of  one 
each),  whereas  a  form  of  nonhomogeneous  model  seems  almost  uni¬ 
versally  q[>plicable  and,  indeed,  provides  significant  improvements  in 
demand  forecasting  accuracy. 

Crawford  [2]  provided  a  comprehensive  description  of  the  variabilily 
in  demand  for  large  sets  of  recoverable  aircraft  spare  parts  and 
pmnted  out  the  implications  of  that  variability  for  readiness  and  sus- 
tamabOity  and  for  spares  requiremmits  estimations  and  capability 
asaeswnent  modeling.  He  also  showed  that  the  numbers  of  items  in 
resupply,  especially  in  the  depot  repair  pipeline,  were  not  only  hi|^y 


nonatationary  atoehutie  praoeu  ia  ataaply  one  in  which  the  arrival  intenaity 
variM  over  tfana.  Thnughout  thia  report,  we  uae  the  terma  nonAomcgeneoiia  and  non- 
atationary  interdtangeahty,  altboaih  in  general  naage  nonxtationary  proceaaea  are  a 
anheet  ef  neahomogeneoga  praceaeei. 


variable  but  on  average  were  bi^Mr  than  aaeumed  in  requbremente 
eatitnatwam  and  oqMbility  tteeeQoment  modeling.  He  also  oonrectly 
pomted  out  Uiat  it  ie  really  the  contents  of  the  resupply  pipeline  and 
its  variability,  rather  than  varialatity  in  demand,  to  wh^  system 
performance  is  most  vulnerable. 

This  research  by  Crawford  [2]  and  the  research  he  had  done  imme- 
diatriy  before  it  reopened  maiqr  ci  the  fundamental  questions  initially 
raised  in  the  eariy  years  by  Brown  and  oUiers.  Crawford  pointed  out, 
for  example,  that  the  demand  models  inoorponrated  in  current  logistics 
management  decision  process^,  most  notably  spares  requiremoits 
estimation  and  capability  assessment  modding,  are  inconsistent  with 
observed  data;  moreover,  the  inconsistencies  miggest  that  we  realty 
don’t  underste^  the  dmnand  process  as  well  as  vm  seem  to  imply  in 
our  (haracterizations  o£  rite  imture  of  the  process,  its  variability,  its 
stationarity,  or  its  relati<mship  to  activity  levd  (e.g.,  flying  hours).  It 
is  important  to  note  for  our  later  discussions  that  Crawford’s  woric  in 
this  case  examined  worldwide  aspirations  of  demands  partitianed 
quarterly.  This  fact  shaped  bis  results  in  important  ways,  as  we  shall 
seebdow. 

Crawford’s  work,  which  was  directed  toward  better  understanding  of 
Ihe  behavior  of  demand  processes  for  aircraft  spare  parts  in  peace¬ 
time,  coupled  with  RAND’s  interest  in  logistic  support  in  wartime, 
provided  much  of  the  motivation  for  an  important  bodty  of  work  done 
at  RAND  in  the  IflSOs,  most  of  it  in  a  mqjor  underlining  of  Prqject 
AIR  FORCE,  which  became  p<^ulariy  known  as  the  Uncertainty 
Project^  The  veiy  title  of  this  woric  reflected  the  thrust  of  Crawford’s 
analyses  and  the  thinking  of  others  at  RAND  that  the  variability  in 
peacetime  demands  for  aircraft  spare  parts  would  be  compounded  in 
wartime  by  system  disruptions,  resource  losses,  and  the  inevitable 
suzivises  of  the  combat  scmiario.  In  the  course  of  the  Uncertainty 
Pnject,  a  set  management  initiatives  emerged  that  was  intended  to 
mitigate  the  effects  of  uncmrtainties  in  resource  demands.  Those 
initiatives  were  given  the  name  CIX)UT  (Coupling  Lt^istics  to 
Operations  to  meet  Uncertainties  and  the  Threat).  The  CLOUT 
initiatives  are  described  by  Cohen,  Abril,  and  lippiatt  [20].  The 
frilowing  discussion  draws  heavily  from  the  Summary  of  their  report 

^The  Unoertainty  Project  waa  part  of  RAND’s  Resource  Management  Program.  It 
was  formally  entitM  “Enhancing  the  Integration  and  Responahreneas  the 
Siqiport  System  to  Meet  PeaceUme  and  Wartime  Uncertainties.*  It  was  spona*^  by 
Hei^qaartars,  USAF^LEX,  and  Headquarters,  AFLCOCP. 
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The  CLOUT  initiativee  axe  wcemplea  from  a  taxcmomy  oi  more 
generic  etrat^iee  fiar  coping  wifri  uncertainty  described  by  Hodges 
[66,  20-24].  Die  uncertainty  is  of  eseentialfy  two  kin^:  (a)  atatia- 

tieal  uncertainty,  defined  aa  variaUlity  observed  in  repeataUe  phe¬ 
nomena,  and  (b)  atate-of-the-world  uncertainty,  defined  as  uncertainty 
that  arises  in  {dienomena  that  are  not  repeataUe,  not  observed  or  ob- 
aervalde,  or  both  [56,  pp.  S-14].  Hanning  for  wartime  is  firau^t  with 
uncertainty  of  the  latter  kind;  in  fact,  state-of-the-world  uncertainty 
can  fairiy  be  said  to  dmninate  the  wartime  scenario  as  wdl  as  mili¬ 
tary  planning  for  it. 

Essentially,  the  CLOUT  initiatives  generally  place  less  reliance  on  a 
liduiess  of  spares  and  take  greater  advanfr^  oi  more  flexible  re¬ 
sources  such  as  maintenance  and  distribution.  That  strategy  derives 
logically  from  the  difficulty  and  cost  of  a  ‘buyout”  strat^y  that  would 
attempt  to  provide  ample  quantities  of  spares,  for  example,  to  allow 
the  qrstem  to  cope  wiffi  the  levels  of  uncertainty  in  demand  that  it 
mi|^t  face,  eqiecially  in  wartime.  At  the  theater  level,  there  are  sig¬ 
nificant  payoffii  to  be  gained  firom  alternative  operating  policies  for 
theater  distribution  systems  that  take  fuller  advantage  of  responsive 
lateral  resupply  and  lateral  repair  options.  Thqy  also  sugg^  that 
doser  coupling  of  the  depot  repair  system  to  the  combat  forces  has 
significant  payoff  in  aircraft  availability.  The  thrust  of  the  thinking 
underlying  CLOUT  is  to  rely  less  on  an  ampleness  of  goods  and  more 
on  management  adaptations.  (Hodges  describes  these  as  examples  of 
passive  and  active  strat^es,  respectively.)  That  thinking  has  impor¬ 
tant  implicati<ms  for  system  design  as  well  as  management.  Many 
characteristics  of  the  current  system  need  to  be  changed  to  achieve 
the  kind  of  relevant,  timely,  and  robust  performance  needed  to  cope 
with  unantidpated,  urgent  demands  for  resources. 

Ihe  magnitude  and  pervasivmi^  of  wartime  and  peacetime  uncer¬ 
tainties  sdso  suggest  the  need  in  both  capability  assessment  models 
and  spares  requirements  estimation  to  take  expUdt  account  of  realis¬ 
tic  levds  of  p^ioetime  and  wartime  uncertainty  and  the  ability  of  the 
support  system  to  cope  with  tiiem  throo|^  the  kinds  of  adaptive  be¬ 
havior  mcemplified  by  the  CLOUT  initiatives.  In  the  face  of  such  un¬ 
certainties,  the  robustness  of  S3r8tmn  perfinmance  in  a  variety  of  sce¬ 
narios  is  at  least  as  important  as  the  level  of  system  performance  in  a 
sin^e,  specified  planning  scmiario. 

The  implications  of  this  approach  for  l<^pstics  management,  logistics 
poliqy  analysis,  and  the  design  of  logistics  management  systems  are 
fiindamenttdly  important.  In  problems  involving  state-of-the-world 
uncertainty,  analjrsts  have  traditionalty  retreated  to  analytic  methods 
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infamAni  to  deal  with  statistical  unosrtamty,  methods  with  whkh 
^ey  are  fiuooiliar  and  oomfiarti^.  Thmldng  deariy  about  the  impB- 
catbiis  of  stateKtf-th»>w(»ld  imeertaiiity  ftr  partieular  arohcations  is 
both  di£Bcult  and  iiwfrmiliiir  finr  many  persons  involved  in  poliqr 
analysis.  Periiaps  the  most  impmrtant  central  message  this  woric  is 
the  need  to  take  ezplidt  account  o£  uncertainty,  particulariy  state-<tf* 
the-worid  uncertainty,  in  fiarmulating  policies  and  designing  qrstems, 
and  to  take  ezplidt  st^  to  ensure  that  the  perfonnance  of  those 
polides  si^stems  is  robust  in  the  face  of  those  uncertainties. 

The  CLOUT  initiatives  have  been  incorporated  into  the  Air  Force's 
new  Logistics  Concept  of  Operaticms.  Ihey  dearly  have  important 
itnpli<»iitifln«  for  spares  and  repair  requiremmits  estimation.  Those 
impHcationa  are  the  prindpal  interest  of  the  larger  project  of  whidi 
thiw  demand  forecasting  and  modeling  work  is  (me  part.  Our  review 
of  past  RAND  researdi  in  demazul  forecasting  has  presumably  made 
it  dear  that  whatever  improvements  mi^t  be  possible  in  demand 
forecasting  should  be  folly  exploited,  even  in  the  face  of  extensive 
employment  of  management  ad^jitations  in  logistics  (operations. 
Ab^  et  aL  [57]  describe,  fin*  ezamide,  a  depot-level  repair  prioritiza- 
ti<m  and  asset  allocation  mechanism  which  is  inten^d  to  be  muifo 
more  adaptive  in  the  fooe  of  uncertain  demands  tiian  are  the  current 
depot  repair  management  and  stock  control  and  distribution  systems. 
As  adaptive  as  it  is,  it  still  depends  to  some  mrtent  on  demand  fore¬ 
casts.  The  need  f(n:  improvement  in  our  ability  to  forecast  was  re¬ 
cently  made  emphatically  dear  by  Lippiatt  [1],  who  observed  large  er¬ 
rors  in  forecasting  NRTS  actions  using  the  traditional  methods  of 
AFLC’s  standard  system.  He  observed  that  the  further  into  the  fu¬ 
ture  the  forecast  was  made,  the  greater  the  error,  periiaps  an  unsur¬ 
prising  result  in  retrospect,  but  the  error  was  consistently  positively 
biased,  a  phenomenon  that  may  have  been  assodated  with  the  time 
period  examined. 

In  spares  procurement,  too,  because  of  the  long  lead  times  frequently 
encountered,  accurate  demand  forecasts  play  an  important  role  in 
shaping  the  mix  of  spares  procurmnent  actions  to  achieve  an  effective 
stockage  posture.  Thus  our  exploration  of  demand  forecasting  and 
modeling  is  not  only  an  important  part  of  our  requirmnents  estima¬ 
tion  research;  it  is  also  important  to  a  variety  of  logistics  management 
dedsionmaking.  We  believe  it  can  improve  such  decisionmaking,  es- 
pedally  over  the  longer  planning  horizons  typically  associated  with 
recoverable  spares  procurements,  and  enhance  the  Air  Force’s  ability 
to  achieve  greater  effectiveness  firom  its  available  resources.  We  now 
turn  (nur  attention  to  the  Air  Force’s  approadi  to  demand  modding 
incorporated  in  the  current  system  for  estimating  spares  and  repair 


ra^mvaieBts  befim  deacribing  in  fidtowing  aectioins  more  effective  al- 
tenu^ivee  to  tiiat  eppcoedi.  A  more  cempleto  deecription  the  cur- 
mt  roeeveraUe  apene  reqoireiBents  qret»  can  be  fiNind  in  Re£  68. 

llEBfAND  MODEUNG  IN  THE  CURRENT  STffnSIff 

A  traditional  apfireadi  ia  taken  to  modeling  demand  by  APMCa  cur¬ 
rant  aparaa  and  rqiair  requiramenta  eatimatkm  ayatem.  llie  wotid- 
wide  mean  demand  rate  tar  eadi  maater  atock  number  ia  ertimated  in 
a  atrai^tforward  way,  item  by  item,  uaing  an  eiid^t^juarter  moving 
average/  Becauae  of  data-collection,  tranamisaion,  and  proceeaing 
timea,  the  demand  counta  availaUe  in  the  central  qratem  are  at  least 
(MM  quarter  bdiind  Thus,  if(me  were  at  a  point  in  time  in  the  middle 
oi  quarter  n,  one  would  ladr  viaibilily  at  the  demands  to  date  in  ({uar- 
ter  n  and  the  demands  that  occurred  in  quarter  n  -  1.  Ihe  demands 
in  quarter  n  -  1  typically  become  availaUe  in  the  central  i^stmn  late 
in  quartern/ 

The  eiid^t-quarter  moving  avmrage  gives  as  much  wei^t  or  impcnr- 
tance  to  demands  eifdrt  (piarters  in  the  past  as  to  demands  only  one 
quarter  in  the  past  Hie  moving  average  does  have  the  advantage, 
however,  of  smoothing  the  demands  across  (piarters  so  that  the  qrs- 
tem’s  estimate  of  demand  rate  is  not  so  volatile  as  it  would  be  using 
less  data.  On  the  other  hand,  it  is  not  as  responsive  to  changes  in 
demand  rates  as  some  alternative  approadies.  The  current  systmn 
assumes  that  demand  rates  per  flying  hour  are  constant;  i.e.,  de¬ 
mands  are  directly  proportional  to  ftying  hours.  Thus  if  flying  hinurs 
are  projected  to  increase  by  x  peromt,  eqiected  demands  will  also  in¬ 
crease  by  z  percent.  One  thrust  of  this  raport  is  to  presmit  alterna¬ 
tives  to  fliese  demand  estimating  tediniques  that  substantially  re¬ 
duce  expected  forecasting  error. 

The  re(iuirements  for  safety  stock  are  computed  by  the  current  sys¬ 
tem  using  an  estimate  of  t^  prrfoability  dMributkm  ci  the  niunbm 
of  itmns  of  each  master  stock  numbmr  fomity  in  resupply  based  on 
steady-state  assumptions,  i.e.,  statkmary  demand  rates,  known  activ¬ 
ity  levels,  constant  pipeline  times,  and  ocmstant  NKTS  rates.  The 
negative  binomial  probability  distributum  is  used  to  describe  fliis 

tyhe  moving  swraso  hM  tiie  fi»m  XD^X^  wham  tbo  {D^|  are  die 

obaerved  over  ci^t  gaartan,  dw  |^)  ue  die  itan  foriof  houre  dwived  ftom 
■B  of  foe  eppliBetteiie  of  foe  itam,  and  foe  aammadona  are  taken  over  die  eiifot  «|iiar- 
tare  of  peat  ea^perience. 

^Aldioa^  die  D036C  ayatem  haa  vwy  current  demand  data  available,  die  D041 
ayatem,  the  ^ratam  that  aupportaraqnirBmentaeetimatkoa,doee  not 
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mdran  variaUe.  The  VTMR  the  diatribotUm  is  baaed  on  anpirical 
obeervations  by  Stevttos  and  IfiU  [S9]  euggeeting  that  the  obaerved 
VTMR  of  donanda  fin-  reeoveiable  q>are  paxta  waa  an  inereaaing 
fiincti<m  of  the  mean  demand  rate,  ^  inereaaing  at  a  decreasing 
rate.  The  observed  VTMR  is  mroneoudy  treated  as  thou|^  it  wore 
the  true  VTMR.  It  has  the  fimn  oi  the  unbiased  estimator  of  the 
variance  divided  by  the  observed  mean  demand  rate,  ^evmu  end 
Hill  chose  a  power  functi<m  on  empirical  grounds  to  describe  the  rda- 
rionahip  of  t^  form  VTMR  «  aa^,  where  z  represented  mean  de¬ 
mand  rate.  At  the  time  of  this  writing,  the  variable  z  refnesenta  the 
mcpected  number  of  itona  of  a  given  type  in  resupjdy,  i.e.,  the  item 
pipeline,  and  the  values  of  a  and  b  are  1.132477  and  0.3407513,  re¬ 
spectively.  If  the  resuhiig  VTMR  is  less  than  1.01,  it  is  set  equal  to 
1.01;  if  it  is  greater  than  5.0,  it  is  art  equal  to  5.0. 

A  priori,  there  is  no  Intimate  basis  fi»r  the  assumption  that  the 
VTMR  i^uld  vary  in  any  systematic  way  with  mean  demand.  Yet, 
the  data,  as  aggregated  by  Stevens  and  Ifill,  do  suggest  such  a  rela¬ 
tionship. 

It  is  not  dear  that  the  power  function  used  in  the  current  system  is 
appropriate.  The  actual  fit  between  calculated  item  pipelines  and  ob¬ 
served  values  of  the  VTMRs  assigned  to  the  probalrility  distributions 
of  those  pipelines  at  the  individual  item  level  is  remarkably  poor. 
Figures  2.1  and  2.2  illustrate  this  ctmtention.  Hie  obeervations  sim¬ 
ply  do  not  fit  the  data,  jnrobably  because  of  nonstationarity  in  the  un- 
d^lying  demand  process  that,  cus  we  will  show  below,  can  induce  a 
profirand  overestimation  of  the  VTMR.  The  use  of  the  power  function 
is  not  on  solid  ground  given  what  we  have  learned  about  the  numeri¬ 
cal  values  of  the  VTMR  estimator  in  the  presence  of  nonstatimiarity 
in  demand;  nevertheless,  it  is  difficult  to  find  a  better  alternative  ap- 
pro^i,  althou^  the  numerical  parameters  of  the  function  can  be  im¬ 
proved  by  accounting  mcplidtly  for  forecasting  uncertainty,  which  the 
current  method  ignores. 

Our  emnments  about  the  current  system’s  anvroadi  should,  perhaps, 
be  more  forgiving  because  the  pi^lem  vi  sensibly  estimating  the 
variance-to-mean  ratio  of  the  distribution  of  the  number  of  items  of 
each  type  in  resuiqdy  is  very  difficult  Sherbrooke  [44,  p.  32]  and, 
later,  Hodges  [56,  p.  32]  suggested  that  numerkalty  estimated  values 
of  the  VTMR  could  be  computed  using  the  method  of  mazimum  likdi- 
hood;  however,  this  aiqirosudi  is  traditionally  q;>plied  on  an  item-by¬ 
item  basis.  The  numerical  values  thid  result  finan  nich  an  sfqnoadi 
sre  hi^y  volatile  and  would  probaUy  result  in  hi^ity  volatile  esti¬ 
mates  of  the  total  number  of  spares  of  eadi  type  the  requiremoits 
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qratraa  eotinuites  riwald  be  in  the  inventory  system.  Such  vrdatility 
is  (me  of  the  principal  cauaee  of  items  being  in  l<mg  supply.  Thus  an 
approach  to  estimating  the  variance  of  the  pnhafaility  distributi<m  of 
the  number  of  itmns  in  resupply  that  is  based  in  some  way  an  aggre¬ 
gations  of  items  <»r  that  em|d<qrs  some  kind  of  smoothing,  rtalnlizing, 
or  filtering  tedmique  seems  to  be  indicated.  Alternatives  to  the  cur¬ 
rent  system’s  method  of  establishing  the  variance  were  evaluated  in 
this  research,  and  we  reoommoid  one  that  rqnesents  a  modest 
imiaevement  to  the  current  apfHoadi,  but  one  that  is  <m  mudi  more 
solid  logical  ground.  The  disiiusshms  in  Section  6  shed  additionai 
Ul^t  cm  tlm  problem  of  variance  estimation. 

Ahhou^  the  principal  thrust  of  our  research  in  the  estimation  of  re- 
(piirements  for  aircraft  recoverable  spares  and  depot  repair  is  to  laing 
explicit  recognition  of  flexibiliiy  and  responsivmiesa  to  the  spares  and 
repair  recpiirements  estimaticm  {nrocess,  in  this  report  we  specifically 
address  issues  of  demand  modding  and  forecasting  in  the  hcqm  of  en¬ 
hancing  the  cost-effectiveness  ct  the  estimation  process. 


3.  S01IEEXPL0RATI0I<«  OF  BASE-LEVEL 
DEBIANDDATA 


Prompted  by  our  increaeiiig  intereet  in  the  implications  at  ui^artainty 
for  ^bctive  ooadMUt  logistics  saf^Mrt,  and  reinfinced  by  the  findings  of 
Crawford  [2]  that  line  items  with  bij^  demand  variability  tend  to  be 
the  troublesmne  ones  in  terms  inventory  system  performance,  we 
recqpmied  some  of  the  demand  modding  issues  originally  raised  by  the 
examination  of  base-level  demand  data  by  earlier  RAND  researdiers. 
The  data  explorations  described  in  this  section  led  to  two  principal 
findings,  eadi  of  which  is  important  to  the  Air  Force’s  ability  to  provi¬ 
sion  its^  cost-effectively  with  aircraft  recoveraUe  spare  parts:  (a) 
There  is  strong  evidence  that  parts  demand  processes  are,  in  general, 
nonstationaiy,  and  (b)  certain  diaracteristics  of  specific  line  items 
whidi  are,  in  many  cases,  known  in  advance  tend  to  be  associated 
with  hi{d>^  levels  of  demand  variability,  lire  implications  of  the  first  of 
these  observations  are  important  for  modeling  the  process  correctly, 
thus  improving  our  ability  to  forecast  demands  and  to  understand  the 
demand  process  somewhat  better  than  we  now  do.  The  second  obser¬ 
vation  is  important  to  our  ability  to  manage  logistics  support  better, 
to  know  when  exceptional  management  action  is  needed,  and  to  im¬ 
prove  our  formulation  and  implemention  of  sensible  policies  for 
spares  investments  and  inventory  system  management. 

The  Air  Force’s  Logistics  Management  Center  (LMC)  at  Gunter  AFB, 
Alabama,  kindly  provided  RAND  with  data  firom  several  bases  firom 
which  we  were  able  to  infer  actual  demands  on  base  supply  by  base 
maintenance  activities.  One  of  the  bases  fix>m  which  data  were  pro¬ 
vided  was  Bitburg  Air  Base,  FRG.  The  Air  Force  has,  for  some  thne, 
maintained  a  wing  of  F-15C/D  cdrcraft  at  Bitburg,  and  we  acquired  48 
weeks  of  transaction  data  firom  Bitburg  firom  which  we  extracted 
transactions,  then  demands,  for  F-15  recoverable  LRUs  and  SRUs. 
The  observations  that  follow  were  made  fi^m  those  data. 

ITEM  CHARACTERISTICS  THAT 'TEND  TO  PREDICT 
HIGHVTMRs 

In  the  Bitburg  analysis,  we  were  able  to  obsmve  monthly  demand 
rates  per  flying  hour  as  weU  as  demands  independent  of  flying  hours. 
The  flying  hour  production  during  the  48-week  period  to  which  the 
data  ai^lied  was  sufficiently  r^ular  that  it  did  not  make  a  substan¬ 
tial  difference  to  these  findings;  however,  the  data  described  only 


montli]^  laying  hour  activity,  not  dai^  faring  hct^  In  eompariiig  tiie 
diatrilmtiosi  of  VTMRb  among  Hne  itema  uting  numthly  demando  and 
months  flying  hours,  it  was  daar  that  tha  naa  of  flying  hours  was  not 
eqpedal]^  impwtant  because  they  did  not  vary  much.  Since  mudi  of 
what  we  learned  derived  fitun  oar  ability  to  obemrve  demands  on  ev- 
eiy  line  item  evmy  day,  we  chose  to  ignore  demand  rates  pmr  flying 
hour  in  favor  of  analyzing  simide  counts  demands.  It  must  suffice 
here  to  say  that  we  are  confident  that  our  observations  would  be  as 
valid  had  we  accounted  for  flying  hours  at  every  step  of  the  analysis 
as  they  are  without  having  dme  so. 

Items  with  large  QPAs  (quantities  per  applkatiim)  or  acti<m  quanti¬ 
ties  greater  than  1  tended  to  have  hi^d^  VTMBs.  These  two  subsets  of 
line  itans  had  a  substantial  intersection.  In  the  case  ci  a  demand 
transaction,  the  action  quantity  is  simply  the  number  of  units  of  issue 
(e.g.,  each,  dozen,  pair,  box)  reported  in  the  transaction. 

In  the  discussion  that  follows,  we  divide  the  demand  data  into  24  two- 
wedE  intervals  and,  for  line  item  j,  cominite  the  observed  mean  de¬ 
mand  rate,  nij,  as  &e  total  numbm*  of  demands  in  the  24  two-week 
periods  for  line  item  j  divided  by  24.  The  VTMR  estimator,  tj,  is  sim¬ 
ply  the  unbiased  estimator  of  the  variance  divided  by  mj.^  lliere  were 
973  items  in  the  sample,  all  F-15  recoverable  LRUs  and  SRUs. 
Demands  that  were  satisfied  by  intermediate-level  repair,  that  is,  by 
maintenance  itself,  were  excluded  firom  this  sample.  The  remaining 
demands  are  those  satisfied  by  the  inventmy  system,  not  by  repair. 
The  distribution  of  (rj)  reflect  75  percent  al^e  0.96,  50  percent 
above  1.03, 25  percent  above  1.61,  and  5.2  percent  above  3.0.  The  dis¬ 
tribution  had  a  very  l<mg  tail  with  some  extraordinarily  hiid^  VTMRs, 
the  hiid^ert  being  a  startling  56.16.  The  unweighted  mean  ci  the 
VTMRs  was  1.79  with  a  remarkably  hi^  standard  deviation  of  3.52. 
Eliminating  all  line  items  with  action  quantities  greater  than  1  or 
QPAs  greater  tiian  4  reduced  the  sample  size  fixnn  973  to  918  and 
dramaticdb^  altered  the  distribution  of  (rj).  Its  unwei^ted  mean  was 
reduced  firom  1.79  to  1.34  and  its  stancford  deviation  fimn  3.52  to 
1.38.  The  number  of  line  items  with  VTMRs  greater  than  3.0  de¬ 
creased  fimn  51  to  16,  and  the  number  ci  line  items  with  VTMRs 
greater  than  5.0  decreased  from  30  to  5.  Thus,  of  the  55  line  items 
eliminated  because  of  large  QPAs  or  action  quantities  greater  than  1, 
35  had  VTMRs  greater  than  3.0  and  25  had  VTBfRs  greater  than  5.0. 
In  percentage  terms,  5.2  percent  of  tiie  original  973  line  items  had 
VTMRs  greatm-  timn  3.0,  and  3.1  percmit  had  VTMRs  greater  than 

^Tbe  onbiaMd  estiinator  of  the  variance  is  shen  Iqr  [n^n  -  1)]  -  (Zxj^n)^], 

where  xj  is  the  number  of  demands  in  the  ith  two-week  period  and  n  *24. 


S.O.  TImm  paremtagM  dacraafled  to  1.7  and  0.5,  napectivdiy,  the 
f  tt  Hna  ttaaM  taaMMung  idler  remoiviag  thoae  with  actum  quantities 
graater  tiiaa  1  or  QPAs  gnuder  tiian  4. 

Several  obeervatkms  emerge  from  these  results.  Blost  important, 
perhaps,  is  the  firct  that  the  distribution  of  observed  VTMRs  (we  are 
talking  about  VTMR  estimates  of  the  form  previous^  described  as  rj 
thnmihMit  Uiis  discussion)  at  base  level  baaed  on  a  two-week  parti- 
tioning  of  the  demand  data  is  dramatically  differmit  from  that  at  ays- 
traiwide  level  as  r^ected  in  the  spares  requirements  database  (ei^t 
quarteriy  observations  of  worldwide  demands).  As  we  will  discuss 
below,  our  partitioiui^  of  the  data  into  two-week  intervals  in  contrast 
to  the  quartnly  intervals  used  in  the  spares  requirements  system  ac¬ 
counts  tar  some  of  this  difference,  but  not  all  of  it  The  differeime  is 
probaUy  amplified  by  the  fact  that  the  Bitilnirg  data  span  48  wedu 
whereas  the  systemwkte  VTMRs  are  computed  using  eijht  quarters  at 
data.  The  implications  of  this  observation  for  demand  modeling  are 
impoitant:  (a)  One  should  not  observe  numerical  values  of  s3rs- 
temwide  VTMRs  and  apply  them  directly  to  base-level  demand  pro¬ 
cesses  as  the  current  spares  requirements  system  does,  (b)  one  cannot 
infer  much  about  the  volatility,  uncertainty,  or  unpredictalrility  of 
base-level  demands  fimn  systemwide  data  aggregated  quarterly,  es¬ 
pecially  if  the  partitioaing  is  different,  and  (c)  there  are  serious  mea¬ 
surement  inoUems  associated  with  the  VTMR  estimator. 

A  second  observation  is  that  action  quantity  (in  this  case  the  number 
at  parts  requested  in  a  requisititm)  acts  as  a  VTMR  multiplier.  If  a 
particular  stochastic  fuooess  is  simide  Pmsson,  then  its  VTMR  is  1. 
If,  with  eadi  requisitum  arrival,  n  demands  occur,  then  the  VTMR  of 
the  dwnand  process  is  n,  Init  the  process  is  no  longer  simple  Poisson; 
it  is  compound  Poisson  (in  this  case  with  a  degenerate  compounding 
diatribuiion).  Thus,  comparing  its  VTMR  to  1  may  tend  to  lead  one  to 
conclude  ffiat  the  process  is  somehow  out  of  control,  wild,  or  unpre- 
dictaUe,  based  on  the  value  of  n,  adien  in  fact  it  is  mmotly  as  pre- 
dictdde  as  a  simide  Poisson  process.  Thus,  the  requisitioning  of  more 
tiian  one  unit  at  a  time  of  some  line  items  causes  the  VTMRs  of  th»r 
demands  to  be  greater  than  the  VTMRs  of  their  requisition  arrivals. 
It  is  important  to  note  that  in  the  Bitinirg  data  only  3.7  pmcent  of  the 
original  973  line  items  (36  line  items)  reflected  any  action  quantities 
greater  Hum  1;  thus  Crawfind’s  and  Lu’s  observations  about  the 
unsuitalnlity  of  omnpound  Poisson  probability  distributions  to  de¬ 
scribe  puts  demand  processes  are  stronidy  reinforced  1^  the  Bitburg 
data.  Recall  our  earlier  quote  firom  Crawford  [54,  p.  32]: 
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Hm  pnetiee  oi  aMuming  an  anival  proceaa  ia  compound  Poiaaon  wiien 
the  dite  axhihit  a  varianea-to-maan  ratio  graatw  than  1,  aa  ia  <^ten  ad¬ 
vocated  in  the  litoatare,  haa  little  kgUiautte  juatifteation.  If  tte  vari- 
ance-to-mean  ratio  ia  aignifieantfy  lati»r  than  1  (whatever  that  meana), 
the  arrival  proceaa  ia  more  oAm  nonhemogeneoua  than  [it]  ia  ccanpound 

Poiaatm - In  invoitory  applkatuma,  treating  the  arrival  laoeaaa  aa  a 

compound  Puaaon  when  it  ia  in  fact  nmihomogeneoua  Ptawn  era  be 
ahovm  to  have  a  Mgwifiwtw*  dfeet  ra  the  optimal  atocking  plan. . . 

Since  our  analyaU  <rf'  demand  data  finun  Bitburg  was  confined  to  de¬ 
mands  for  recoverable  items,  it  is  not  surprising  that  so  few  line  items 
experienced  action  quantities  greater  than  1;  the  Air  Force’s  reorder 
policy  for  recoverable  items  implemented  in  the  standard  base  supply 
83rstmn  is  (s,  s  -  1);  ie.,  when  Ae  number  of  assets  on  hand  plus  due- 
in  minus  due-out  falls  below  the  stock  level,  s,  an  order  is  placed  im¬ 
mediately  (usually  daily,  as  a  practical  matter)  for  replenishment. 
This  is  commonly  described  as  a  continutms  review  reorder  poliqr 
with  an  order  quantity  of  1.  For  items  with  action  quantities  greater 
than  1,  the  compound  Poisson  model  may  be  appropriate  to  accommo¬ 
date  the  hitler  variance  that  trade  to  be  associated  with  these  items. 
A  large  QPA  does  not  necessarily  imp]^  a  compound  demand  distribu- 
tum.  All  we  know  firom  the  Bitirarg  data  is  that  large  QPAs  tend  to  be 
associated  with  high  demand  varialnlily.  As  we  pointed  out  above, 
the  set  of  items  with  action  quantities  greater  than  1  and  the  set  with 
large  QPAs  ovralap  significantly.  Of  the  36  line  items  reflecting  ac¬ 
tion  quantities  greatra  than  1  in  the  Bitburg  data,  only  six,  i.e.,  16.7 
percent,  were  known  to  have  a  QPA  of  1.^  These  observatiras  about 
action  quantities  simply  suggest  that  large  VTMRs  do  not  necessarily 
imply  unpredictability' 

After  eliminating  the  items  with  action  quantities  greater  than  l  and 
QPAs  greater  than  4  fixim  the  distributira  of  observed  VTMRs,  frj), 
there  were  still  five  of  the  remaining  918  line  items  with  VlidRs 
greater  than  5.0,  the  uppmr  bound  estaUished  for  VTMRs  used  in  the 
spares  requirements  estimation  process  to  describe  the  variability  of 
the  number  (rf*  assets  of  a  given  line  item  in  resupply.  Their  VTMRs 
were  5.5,  6.5,  7.2,  18.7,  and  34.6.  Thmr  QPAs  were  1,  3,  3,  4,  and  1, 
respectively.  The  first  three  were  SRUs,  the  latter  two  LRUs. 


^Tbe  QPA  (quantity  per  application)  of  an  item  is  the  quantity  of  the  item  installed 
in  the  beat  hi^er  assembly.  We  hastm  to  add  ftr  the  sake  of  oompleteneas  that  the 
QPAs  of  two  of  the  36  items  were  shown  in  the  D041  ^qdkation  file  as  0.  One  of  them 
was  an  engine  vane,  very  likely  an  iton  with  a  large  (^A;  tte  other  was  an  ezpennve 
switch  in  the  5841  propwrty  daas  vdiose  true  (^A  is  still  unknown. 


The  hii^iest  VTMR  was  associated  with  the  nose  landing  gear  strut,  a 
$26,500  component  whidi,  along  with  one  of  its  principal  SRUs,  the 
$15,600  nose  landing  gear  piston,  exhibited  ratiher  remarkable  de¬ 
mand  behavior.  The  demands  for  the  strut  and  piston  observed  in 
eadi  of  tlie  24  two-wedc  periodB  were: 

Strut  12220420100110100121  51  00  1 

Piston  00000000000000000001550063 

It  is  interesting  to  note  that  the  demands  for  the  large  quantities  of 
both  items,  51,  55,  and  63,  were  all  backordered  with  no  correspond¬ 
ing  MICAP  event,  althoucdi  over  the  entire  period,  there  were  10 
IdICAPs  reported  on  the  strut  but  none  on  the  piston.  Our  assump¬ 
tion  about  thin  wildness  is  that  it  is  the  result  of  some  kind  of  policy 
intervention,  perhaps  a  time  change  replacement,  perhaps  an  inter¬ 
vention  of  the  IM  or  SPM,  compliance  with  a  tedmical  directive,  or  a 
safety-of-fli|^t  consideration.^ 

It  is  not  siuprising  that  three  these  line  items  with  hig^  VTMRs 
(6.5,  7.2,  and  18.7)  have  QPAs  greater  than  1  (3,  3,  and  4).  The  item 
with  the  VTMR  of  18.7,  an  $1,131  SRU  in  the  1440  federal  stock 
class,  experienced  186  demands  during  the  48  weeks,  not  one  of  which 
was  satisfied  tibatmgdi  an  issue  by  base  supply.  Of  the  186  demands, 
86  were  satisfied  by  WRSK  withdrawals  and  the  remaining  100  were 
backordered.  No  MICAP  was  ever  reported  as  a  result  of  shortages  of 
the  item;  it  had  no  apparent  impact  on  mission  capabilily  except  that 
it  presumably  depleted  the  WRSK. 

The  remaining  line  item  hardly  deserves  attention.  It  experienced 
only  nine  demands  in  the  24  two-week  periods.  It  has  such  a  high  ob¬ 
served  VTMR  because  seven  of  the  demands  occurred  in  one  period. 
All  were  satisfied  by  off-the-shelf  issues. 

THE  VABIANCE  OF  THE  ESTIMATOR,  Tj 

One  question  that  is  prompted  by  Bn  investigation  like  this  is  the  role 
of  chance  in  producing  particular  values  of  observed  VTMRs  given 
some  numerical  value  of  tihe  VTMR  underlying  a  stochastic  process. 
This  question  is  especially  relevant  to  the  present  discussion  simply 
because  the  variance  of  the  VTMR  estimator,  rj,  is  quite  large.  A 
VTMR  of,  say,  1.5,  coupled  with  a  mean  demand  rate  of,  say,  0.5  de- 


^If  this  was  the  case,  the  peak  in  demands  would  have  been  a  one-time  ocennrenee 
and  shoud  not  have  been  rrflected  in  estimation  of  the  item’s  future  demand  rate. 


nniiMia  per  poriod  over  eight  periods,  is  likely  to  yidd  an  observed 
VTMR  greater  than  3,0  about  3  pereent  of  the  tuM. 

It  may  provide  some  intuiti<m  about  these  lutooesses  to  recaU  tiiat,  in 
the  Bitiburg  data,  about  5.2  porcent  ci  the  973  original  observed 
VndRs  were  greater  than  3.0.  After  removing  the  55  line  items  with 
action  quantities  greater  than  1  or  QPAs  greater  than  4,  only  about 
1.7  percent  ci  the  918  remaining  observed  VTMRs  were  greater  than 
3.0.  For  these  918  items,  the  distribution  of  observed  VTMRs  is  pm*- 
fectly  oonsistant  with  an  assumption  that  all  the  parts  have  an 
underlying  VTMR  of  1.5  or  less. 

It  is  dear  from  our  observations  d  large  numbers  of  random  realiza¬ 
tions  of  these  demand  processes  as  well  as  the  theoretical  work  by 
Hodges  [60]  that  the  error  distribution  of  the  VTMR  estimator  itself 
bears  much  of  the  responsibility  for  the  distributions  of  observed 
VTMRs  we  see  in  parts  demand  processes.  The  estimator  is  also  very 
sensitive  to  mmstationarity  which,  as  we  discuss  in  Section  6,  also 
contributes  significantly  to  shaping  the  observed  VTMRs. 

nonstahonabity  in  base-level  demand  processes 

In  our  analysis  of  the  Bitburg  data,  we  tried  partitioning  the  48 
weeks  of  data  in  several  ways:  24  two-week  periods,  12  four-week,  8 
six-week,  6  ei^t-week,  and  4  12-week  periods.  We  found  a  remark¬ 
ably  systematic  relationdiip  between  the  observed  VTMRs  and  the 
coarseness  of  the  partitioning.  Tables  3.1  and  3.2  show  the  results  for 
all  parts  and  for  the  subset  of  918  with  QPAs  less  than  5  and  action 
quantities  of  1. 

If  a  process  is  stationary,  the  expectation  oi  the  observed  VTMR  is  in- 
dqiendent  of  the  partitioning,  altimutd*  its  variance  increases  witii 
the  coarseness  of  the  petitioning.  In  the  stationary  case,  the  VTMRs 
in  Tables  3.1  and  3.2  would  be  approximately  equal  within  each  table, 
indepmident  of  the  partitioning.  In  the  actual  case  being  examined 
here,  where  we  pool  the  results  of  alternative  partitionings  of  over 
900  random  realizations  of  demand  processes  a^  observe  such  sys¬ 
tematic  variation,  it  semns  very  hkely  that  nonstationarity  is  in- 
vdved,  owuig  simj^  to  the  lack  of  any  other  explanation.  Wlule  this 
systematic  variation  <k>es  not  prove  nonstationarity,  it  strangty  sug¬ 
gests  it.^ 

^Enitiding  itamc  wiUi  action  quanUtiea  gnelbiir  than  1  or  <^Aa  greater  than  4 
hnran  the  v^naa  ahown  in  TaUe  3.1,  b«t  the  pattnrn  remains  laigaly  the  same  as 
shown  in  TaUe  3.2. 
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TBbl«S4 

Bfibeto  of  lHuitttaak«  fltt  CMhMTVvd  Vnite 
(•U  978  parte) 


No.  of 
Periods 

Weeks  in 
Period 

Aeg.  Observed 

vniR 

95  Percent 
Confidence  Interval 

24 

2 

1.76 

(L88,  1.90) 

12 

4 

1.91 

(L68,  Z16) 

8 

6 

1.92 

(L68,  2.16) 

6 

8 

2.11 

(1.76,  2.46) 

4 

12 

2.20 

(1.85,  2.55) 

Table  8J 

Effecto  <rf  Partitioning  im  Obeervod  VTMR» 
(918  parte  with  QPAs  leas  than  5  aod  aetton 
qnaatitieaof  1) 


No.rf 

Periods 

Weeks  in 
Period 

Avg.  Observed 
VTMR 

95  Percent 
Confidence  Interval 

24 

2 

1.35 

(1.26,  1.44) 

12 

4 

1.41 

(1.32,  1.50) 

8 

6 

1.45 

(1.35,  1.55) 

6 

8 

1.49 

(1.39,  1.59) 

4 

12 

1.60 

(1.47,  L73) 

The  demand  modeling  methods  that  we  will  discuss  in  the  sections 
that  follow  derive  much  of  their  power  from  the  fact  that  they  e]q>lic> 
itly  model  the  nonstationarity  in  the  demand  process  and  separate  its 
effects  fix>m  those  of  the  inherent  variability  of  the  process  around  its 
nonstationaiy  location  parameter.  Before  introducing  these  alterna¬ 
tive  methods,  let  us  first  reflect  briefly  on  the  implications  of  these 
discussions. 

DfFUCATIONS  OF  THE  BASE-LEVEL  ANALYSES 

One  may  wonder  whether  we  can  prudently  infer  as  much  as  we  have 
inferred  finm  data  collected  at  a  sin^e  base  on  a  sin^e  weapon  etys- 
tem.  Sudi  a  concern  is  well  founded  and  suggests  the  need  frv  ext^ 
si<ms  to  these  analyses. 

Our  principal  observations  inferred  firom  the  Bitburg  data  are: 

•  The  obsmved  VTMRs  associated  with  parts  demands  at  base  level 
are  very  different  fimn  those  of  quarterly  demands  aggregated 
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woridwide  because  demands  per  flying  hour  teml  to  be  positively 
oorrdated  across  bases. 

•  Lacgs  QPAs  and  order  quantities  greater  than  1  oontrfliute  sig* 
mficantiy  to  hi^  observed  VTMRs. 

•  The  variance  cf  the  VTMR  estimator  explains  to  a  substantial  ex¬ 
tent  the  occurrence  of  large  values  of  dMerved  VTMRs. 

•  Parts  demand  processes  appear,  in  gmeral,  to  be  nonhomogeneous 
rather  than  compound  processes. 

It  is  important  that  we  distinguish  our  inferences  about  demand  un¬ 
certainty  on  the  basis  of  woridwide  aggregatimis  of  quarteriy  de¬ 
mands,  fmr  examine,  from  tiiose  that  we  make  fr<»a  (^Merving  base- 
levri  demand  processes.  We  caution  tiie  reader  that  our  comparison 
of  tbe  distributions  of  VTMRs  at  the  base  and  i^ystem  levels  here  is 
flawed  by  the  fisct  that  the  base-level  data  span  only  4fl  weeks, 
whereas  the  wmidwide  data  in  the  spares  requironents  database 
span  two  years;  the  periods  of  aggregation  are  different  as  well,  as  we 
have  pointed  out  repeatedly.  Ihe  levels  of  variability  implicit  in  dis¬ 
tributions  of  observed  VTMRs  are  quite  differait  for  &e  two  echdons. 
It  is  also  important  to  note  that  the  item  characteristics  (QPA  and  or¬ 
der  quantity)  that  help  explain  high  observed  VTMRs  at  base  level 
may  also  be  helpful  in  ex|daining  hi^  VTMRs  systemwide. 
Quarterly  partitioning  of  demands  aggr^ated  worldwide  can  produce 
very  different  distributions  of  observed  VTMRs  from  those  observed 
at  base  level  because  bases  are  not  independent,  their  covariance 
terms  typically  being  positive.  Hie  demaid  processes  of  many  parts 
are  nonstationaiy,  so  if  the  worldwide  data  are  partitioned  quarterly 
and  the  base-level  data  are  partiticmed  more  finely,  the  worldwide 
data  will  exhibit  higher  VTMRs. 

We  make  the  observation  that  these  demand  processes  tend,  in  gen¬ 
eral,  to  be  nonhomogeneous  rather  than  compound  processes  because 
of  the  sensitivity  of  the  distribution  of  dliserved  VTMRs  to  the  coarse¬ 
ness  of  partitioning  of  the  demand  stream,  and  bee:  «  of  the  small 
number  of  requisitions  with  order  quantities  grea  than  1.  This 
observation  itself  is  important  to  our  efforts  to  develop  more  rational 
models  of  the  demand  process  that  wiU  yield  improved  demand  fore¬ 
casts  and  more  effective  investments  in  the  face  of  resource  con¬ 
straints.  In  the  several  sections  that  follow,  the  effectiveness  of  tak¬ 
ing  explicit  account  of  nonstationarity  is  clearly  demonstrated. 


It  is  important  to  omphasiwi  again  that  the  observatkms  diseiused 
h«e  appi^  to  peaeatfane  dsoumd  dsta,  not  necessarily  to  demand  pro- 
cesses  in  smrtinie,  not  becanae  the  nnderi^ying  fidlure  process 
^  naoeaaarilyt  but  mix  of  demands  tho  system  sees 

from  units  engaged  in  combat  is  likdy  to  ksdi  (piite  different  from 
peacetime  demands  for  many  reastms.  Our  uncertainty  about  peace¬ 
time  demands  is  likely  to  be  compounded  in  wartime  in  ways  that  we 
canm^  readily  foresee.  Many  events  that  are  essentially  unknowaUe 
in  advance  are  more  likely  to  occur  in  wartime  and  generate  demands 
to  which  tile  logiatics  sivton  must  be  able  to  respond  qui«ddy. 

The  notion  of  atme<4f-the-w(xid  uncertainty  and  its  distinction  firom 
statistical  uncertainty  is  the  kernel  rf  the  issue  here.  Mortitfthedis- 
euanmi  in  tins  r^xnt  pertains  to  imcmrtahdy  of  the  statistical  variety. 
WMke  admittedly  important  to  effective  and  effesmt  peacetime  sup¬ 
port,  we  do  not  fully  uttdentand  the  implications  of  stake-of-the-world 
uncertainty  for  wartime  sui^pml  The  proUem  of  better  understand¬ 
ing  those  imidkations  for  combat  logistics  si^ipcnt  is  undoubtedly  im¬ 
portant  and  deserves  further  research. 


4.  ALTi»NATIVEAPPBOACHES  TO  DEMAND 
FOBECA8T1NG 


As  we  aaw  in  our  review  (rf  past  BAND  research  on  the  ttqpic  of  model¬ 
ing  the  dnaands  for  aircraft  spare  parts,  the  early  researdierB  tended 
to  think  ci  these  demand  processes  as  statumaiy,  Le.,  as  being  ran¬ 
dom  observatkms  about  an  unknown  omstant  Recall  that  tiiere  were 
muMroua  reiiireacea  to  the  idea  of  "the  tnm  mean”  in  that  early  liters 
store.  The  ideas  we  wffl  introduce  in  the  discuasion  that  follows  r^ 
resent  a  diffiarent  my  of  thinking  dl)oat  these  stochastic  procoBBOB,  as 
time-vaiyfaig ornoimtatiooaiy.  Nmstatkmarily (or nonbomogeneity) 
cue  produce  large  varianoee  (therefore  large  VTMRs)  even  when  the 
process  at  any  given  time  or  at  apy  given  base  might  he  simide 
Poisson  around  a  changing  mean. 

Any  mnissum  of  an  important  mqdanatoxy  variaUe  in  the  demand 
modding  inrocess  results  in  varialnlity  in  the  mean  being  lumped  with 
the  variabilxty  about  tlm  mean.  As  a  laaetical  mattmr,  it  mig^  be  too 
costly  to  collect  data  related  to  every  possible  explamftory  variable; 
thus  one  postulates  a  reasonaUe  fiuhire  model  and  collects  data  re¬ 
lated  only  to  the  variaUes  in  that  modd.  Many  potentid  explanatory 
variables  are  not  eqdicitly  omsidered.  It  is  important  to  note,  too, 
that  data  ooUectkm  itself  may  amtribute  significantly  to  the  variance 
we  observe  in  the  process  throud^  errors  in  cdlection,  transmission, 
or  laooessing.  As  we  saw  in  Section  3,  the  design  of  the  data-cdlec- 
tion  and  rqmting  system  may  itself  induce  hi^mr  observed  variabil¬ 
ity  in  demand  owing  to  the  omstraint  it  fdaces  on  our  alrility  to  parti¬ 
tion  the  obearvatitms  more  findy.  Thus  the  variability  we  observe  in 
the  denmnd  process  may  have  sevmral  componmits  that  are  not  explic- 
itiy  identifiaUe.  While  we  msy  not  be  able  to  isolate  the  contributicm 
of  every  foctor  to  the  variance  we  observe  in  the  demand  process, 
there  are  gains  to  be  made  firom  separating  the  variance  inherent  in 
the  demand  process  fiom  that  in  its  undedying  location  parameter, 
'nie  precise  functional  form  of  the  relationship  between  demands  and 
flying  hours  as  wdl  as  certain  other  explanatory  variaUes  is  dis¬ 
cussed  in  i^pendix  B. 

The  spare  parts  demand  forecasting  literature  is  replete  with  ideas 
for  using  other  measures  ^  activity  to  predict  demands.  Changes  in 
the  Air  Fmee’s  data  systems  mi^t  also  fifolitate  improved  ttomanH 
forecasting.  In  addition,  maiqr  authms  have  looked  at  parts  foilures 
and  found  them  to  be  readily  explicable  in  retrospect  (retrodictitm 
being  an  easier  task  than  lae^Gtion).  'nieproUemisthat,evenifwe 
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uDderrtood  the  effiKtc  of  auaqr  Wndi  of  omnto  on  the  demand  ivo- 
ooM,  sndi  ttndmtandiag  ient  neoAil  nalMs  we  can  predkt  the  events 
tiMBmeleea.  It  is  impostant  to  aeoount  explicitly  fm  uncertainty  as 
mudi  as  possible.  By  this  we  do  not  mesn  to  ooUect  every  possible 
eqdanatoty  vaziaUe  or  put  a  stress  messnrement  device  on  every 
part.  We  mesn  to  incorporate  a  reahatie  assessment  of  our  uncer¬ 
tainty.  The  alternative  fisrecastiiig  mettiods  we  diacuss  in  tiiis  section 
fiscilitate  exjdkation  of  our  uncertainty. 

It  may  be  hetyAd  to  try  to  dmraeterise  tiie  nature  and  aources  of  un¬ 
certainty  in  die  demand  process.  So  Car  we  have  loosdy  lumped 
together  nonstationaiity,  stodbastic  variation,  omittfil  emdana- 
tory  variables.  In  tihe  diaeitsrion  that  fiillows,  we  identity  types  dT 
uncertainty  that  may  affect  the  demand  process:  ezdumgeafaility 
uncertainty,  modd  and  structural  uncertainty,  dida  uncertainty,  and 
coeffidort  and  stodiastic  variance.^ 


The  eichangeahility  judgment  is  a  critical  foctor  in  the  replication  of 
any  forecasting  method.  To  get  some  dear  intuition  about  this  judg¬ 
ment,  we  must  mentally  separate  out  some  of  the  other  sources  of  un¬ 
certainty.  Considor  a  model  that  is  perfect  in  the  sense  that  we  know 
aU  the  coefficfents  and  relatumdiips  exactly,  without  the  uncertainty 
of  statistical  estimation.  The  key  qpiestimr  is  whether  sudi  a  model 
would  be  a  good  model  of  the  demand  process  in  the  future.  If  we  be¬ 
lieve  the  future  Air  Force  will  be  mudi  like  today’s  Air  Force  with 
similar  suipmt  structures,  polides,  and  strategies,  our  judgment  may 
be  that  our  current  model  of  the  demand  jncoess  may  be  exdiange- 
dde  with  a  modd  of  tiie  demand  process  in  tiie  feture.  On  the  other 
hand,  the  Air  Force  mi^  adopt  a  suiport  structure  with  only  two 
levds  of  maintenance  or  introduce  tedmological  change  that  could 
substantially  affect  the  repairable  gmieration  inrooess.  Responses  to 
exogenous  shocks  to  the  system  such  as  mi^t  be  felt  in  wartime 
could  also  induce  changes  in  operating  pdides  or  procedures  that 
could,  in  tom,  affect  the  demand  process.  In  such  cases,  our  ex- 
diangeability  uncertainty  would  be  substantial  and  some  explidt  ad- 
justmoit  to  our  frawcasting  models  mid^t  be  needed. 

Altiioug^  reassuring  oursdves  about  the  exdiangeability  is  intrinsi¬ 
cally  difficult,  there  are  smne  things  we  can  do  to  try  to  enhance 
exchangeability.  When  vre  build  models,  we  tiy  to  indude  variaUes 

^Snvral  of  Um  idiM  pnMDtod  in  Uus  nctian  an  firam  oor  RAND  ooUesfUM  David 
Draper  and  Jamaa  8.  Hodsta. 
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wMmpiw,  me  wiild  wat  to  indadb  flying  how  m  an  iplMMtktt  of 
demand  enwi  if  haura  dan%  vaiy  aendi  in  tiia  duat  term 
bacnuae  we  may  need  ta  foracaat  dwaanda  ia  panada  af  increaaad 
aethdigr.  We  da  iUa  beeaMae  we  frd  aMne  oomtetdbla  abaot  tlie 
appliratiwi  of  the  medd  to  diWhwit  aitnalHiia  if  aaodai 
rariabke  diat  we  tiniik  will  erplain  the  ^fibraioe  betwean  the 
preeamt  and  die  ftitare. 

Wham  eadiangeafaility  uncertamty  ia  large,  we  tend  to  think  of  it  as 
atate-of-the-worid  uncwrtain^  [663.  When  it  ia  ondl,  we  hope  to  ae> 
otnninodate  it,  at  least  cn^dy  or  paitialty,  witii  the  kinds  of  models 
we  diacaas  in  this  aectum. 

MODEL  AND  SlltllCTIJlIAL  lJN€»liTA^^ 

We  win  radar  to  the  three  kinds  oi  model  unomtainty  as  variaUe  ae- 
lecti<m,  variaUe  qiedfication,  and  structural  uncertainty. 

Variahle  selection  uncertainty  ia  the  uncertainty  introduced  by  our 
inalnlity  to  know  exactly  and  exhaustively  which  variaMes  should  be 
used  to  forecast  the  demands  for  a  particular  part.  This  would  in- 
dude  the  uncertamty  frmn  our  inability  to  use  certain  desiraUe  vari¬ 
ables  because  lack  of  data  on  those  variables. 

VariaUe  specification  uncertainty  is  our  unoortainty  about  exactly 
how  each  variable  should  be  incnrpmated  or  specified  in  tbe  model 
For  example,  should  flying  hours  be  bransformed  into  quadratic  or 
logarithmic  fisrm  or  incorporated  linearly? 

Structural  uncertainty  is  the  micertainty  in  how  various  relationdups 
interact  to  inoduce  the  observed  outccnnes.  For  examide,  does  the 
NBTS  rate  depend  on  the  demand  rate?  Do  the  bases  NETS  more 
parts  during  periods  of  stress?  Rule-versus-model  uncertainty,  a  spe¬ 
cial  case  of  structural  uncertainty,  addrooooo  wbrther  certain  rules  or 
interventions  take  place  fiiat  are  not  captured  by  the  model.  For  ex- 
amfde,  if  the  Air  Fmoe  dianged  its  branspmtatioa  policies  in  a  way 
that  dramatically  increased  item  ptydines,  th^  would  differ 
markedly  frnn  wfaat  our  models  suggest 

DATA  UNCEBTAINTT 

Data  unosrtsinity  is  the  uncertainty  tiiat  results  frnn  errors  in  record¬ 
ing,  tmsnuttbaf,  or  pwrassing  data.  It  is  the  uncertainty  that  re- 
si^  fiuBi  lost  wisgnetir  tspss  and  corrupted  d^a.  It  is  the  oonee- 
quaace  ofrepordngomisMonsdiat  may  wdl  be  the  result  of  necessary 
drcumventiona  of  stmidard  procedure.  As  is  the  case  with  the  nnis- 


iiw  of  iwytoiit  ooriobioo,  tiris  is  m  dillknlt  kibad  of 

oMkIiaiiy  toiasosposatsoi^tfitchsnpifrotpotiBdtoporiod.  For 
Off— ipis,  a  afStaSHtie  and— —portkig  of  4]dac  hours  hf  10  psrcont 
doss  aofc  pMssMt  BMMh  of  a  probhas  fir  dianand  firoosatinK.  Pro- 
saasabfy  tihs  qntan  wfil  jast  sad  np  with  a  dsmand  rate  Uad  is 
hjjhsr  than  the  **trae*  dwsnd  rats.  This  wffl  laouH  m  mo— or  lass 
ths  apprspriats  dsriiions  Tha  pnfaiem  is  rhsn  ths  data  errors  vary 
firoBo  period  to  period,  iwdtiriwg  additional,  inaapHcahle  ffarialalily. 


Another  soutm  of  uncertainty  is  oo^lemnt  variance.  Tliis  is  the  un¬ 
certainty  that  results  from  estimating  coefficients  in  a  modd  statisti- 
caljy  as  <ypeaed  to  knowing  them  exactly.  For  example,  in  the  tradi¬ 
tional  regression  modd,  y  a  a  -i-  hx  e,  coefficient  variance  would  be 
the  consequence  of  our  imt  knowing  the  numerical  value  of  b  a  priori 
In  many  ^tpKcations,  tins  is  the  only  source  of  uncertainty  that  is  ex- 
dicitly  considered  and,  ironically,  it  isdten  the  least  important. 

STOCHASnC  VABIANCB 

The  find  source  of  uncertainty  is  the  stochastic  uncertainty  resulting 
fimn  a  laedictKHi.  Even  if  the  probability  of  a  coin  coming  up  heads  is 
known  to  be  exactly  1/2,  there  is  stodhastic  uncertainty  about  the  out- 
onne  of  any  individud  toss.  In  the  repression  modding  omtext,  this 
correqmads  to  another  draw  firom  the  error  distribution  (the  epsilon 
in  the  eqnesdon  yaa  +  bx-i-e)fi]ra  future  observation.  The  com¬ 
bination  of  parameter  and  stodiastic  variance  is  sometimes  called 
predictive  variance.  The  current  system  ignores  parameter  variance 
and  aU  other  aenrces  of  varidality,  incorporating  only  an  estimate  of 
stodiastic  vaiimiee  as  its  repres— tstion  of  uncertainty. 

ACXXllTimNG  IT«  llNCBir^Al^^^ 

The  sum  of  all  these  sources  oi  variability,  excqit  the  tradttumd  pa- 
noneter  and  stochastic  variance,  makes  up  tiie  nonstationarity  that 
we  see  in  parts  demand  procosses.  The  bulk  of  it  results  from  red 
causes  that  we  do  not  understand  a  priori  or  do  not  measure. 
Althoui^  we  cannot  indude  all  d  these  sources  d  uncertainty  o^c- 
Uty  in  our  modd  d  the  demand  process,  we  can  attempt  to  quantify 
the  magdtndedtfaeir  variability  in  our  modeto  and  decisions.  Aswe 
win  show,  fisrarasHng  dn»vadiss  that  try  ta  account  far  as  many  d 


efhetive  ttoi  the 


Our  idbitttgF  to  aocomt  ftr  thooo  oooibm  of  voriobitit]^  u]Mn 

tinoowB  of  tfaoif  to  tho  munwioo  of  tlio  prooooo  ranaoiii* 

iagxaii^UtyooMtaiiAovortme.  Tlao  io  on  o»fhoaNP>bi1Hgr  jadgmoiit 
in  tiko  varitfkoo  It  is  poss&lo,  for  ermfir,  *%i»  mnsti  of  n 
praesss  is  drifttng  ovtf  tune  bv^  the  varisnoe  of  that  moeesoent  is 
oQBStsttt.  TIm  we  fftfisMfr  in  this  softinn  iacoipovate  tbas 

logic,  akhot^^h  it  is  by  no  mesas  wdBl  fenndeii  For  wuiinple,  there  is 
eooy  reason  to  believe  that  demand  variobility  will  increase  radkalljr 
dnrii^  wsrtune.  Onr  prineqMl  argnsoent  is  suaply  titmt  any  attempt 
to  jncorporate  nonstikionaniy  is  probably  better  than  assnmii^  thrt 
it  does  not  exist;  however,  trying  to  ineorporate  nonstidionarity  in  the 
vmianee  adds  a  layer  of  oookplexity  that  is  pralMdily  unwvranted. 

An  added  bonus  from  inoorpmating  nonstationariiy  eqtUcitly  in  our 
models  of  tiie  demand  process  is  that  we  then  have  a  hand^  knob  to 
turn  when  we  fsce  foture  uncertainty  that  is  larger  than  usual.  For 
example,  in  the  transition  to  wartime  we  could  increase  our  estimate 
of  the  sum  of  these  variance  componmits  to  make  the  system  more  re< 
sponsive  to  the  first  incoming  wartime  data.  We  may  not  be  aUe  to 
forecast  the  directimi  of  change  in  mean  demand  rate  in  a  particular 
set  of  drcumstanoes,  but  we  may  be  aUe  to  increase  our  estimate  of 
the  variance  foitly  easily  to  represent  our  increased  uncertainty  about 
the  future.  We  should  do  evmything  oT  this  kind  that  we  can  before 
we  tiuow  up  our  hands  and  di^  that  no  finecasting  of  the  future  is 
possiUe. 

THE  KALMAN  FILTER 

We  begin  our  discussiim  of  alternative  approaches  to  demand  fore¬ 
casting  with  the  Kalman  filter,  e  model  that  expliddy  incorperates 
nonstationarity  by  treating  the  mean  of  the  demand  laocess  as  a  ran¬ 
dom  variable.  mean  need  not  be  a  simple  ramkan  variable  with 
the  Kalman  filtm;  it  mi^  also  be  a  somewhat  more  complicated 
model  akin  to  a  description  of  the  mean  as  in  a  regressiim  fbnction. 
In  this  section  we  will  mq)lore  the  underlying  the  Kahnan  filter. 
Even  the  simplest  Kalman  filters  have  two  important  dmractmistics 
that  make  them  very  powerful  in  representing  tlw  particular  and  spe¬ 
cial  features  of  parts  demand  processes:  (a)  They  eqtUcitly  model  the 
location  parameter,  or  mean,  ^the  process  as  a  random  variable,  and 
(b)  in  exacts  the  manner  raggeated  by  Kenneth  Arrow  [28],  t^r  not 
only  give  greater  wd|^t  to  more  recmit  observations  than  to  <dder 
mies,  they  also  assign  greater  uncertainly  to  forecasts  in  the  diafamt 
future  tlum  to  those  in  ahiatar  idanning  horixcms,  an  intuitivdy  ap¬ 
pealing  notum. 


Kafanaa  an  an  aapedaUjr  ridi  dais  of  models  ot^paaBy  intro- 
dneed  in  tbs  elactrical  Htsndmre  by  K*h»**«  and  Buqt 

Etl].  Hanes  the  mm— ^ftsr  as  oppoaod  to  the  more  colloquial  pndie- 
tor.  They  han  aineo  grasm  in  braadth  of  iqi|dieation  to  many  fMds, 
e.g.,  emtnt  thaofy  (enginaatingX  stochastic  parameter  regreerion 
BMdels  (eeanomies  and  finance),  and  Bayesian  predictive  methods 
(statiattesX  The  view  of  the  KsAuMn  filter  underiyii^  its  exposition 
here  derives  from  Bayesian  statistics  [^].  In  their  apfdkatiim  to 
madding  demands  fig  aircraft  qmn  parts,  Kahnan  filters  area  natu- 
ral  extenskn  to  Bayesian  statistics  in  die  sense  that  the  probability 
distributioas  of  the  parameters  underiying  the  demand  process  are 
updated  with  eadi  new  observatton.  As  we  will  discnss  at  somewhat 
greater  length  below,  the  Kalman  filter  also  has  the  important  char¬ 
acteristic  that  its  perfigmance  is  fairiy  robust  with  respect  to  its  un¬ 
deriying  asmimptiona.  In  many  aiqiilicationa,  Kalman  filters  have 
been  observed  to  perfigm  better  than  expected  when  their  assump- 
ti<m8  axe  not  met 

In  the  ranaindw  of  this  section  we  introduce  a  simple  Kalman  filter 
modd  and  examine  its  properties.  Then  we  introduce  a  more  general 
modd  and  diseuss  an  example  of  its  ^pfdicatkm.  We  wiU  then  discuss 
a  simpfified  version  of  this  i^fgoadi,  wei^ted  regression,  that  is  eas¬ 
ier  to  implement  and  show  the  results  of  our  evaluations  of  a 
weii^ted  regression  amgoadi  to  demand  figecasting  in  terms  of  tra¬ 
ditional  statistical  measures  of  forecasting  error.  Finally,  we  demon¬ 
strate  its  cost-effectiveness  in  estimating  spares  requirements  in 
terms  of  aircraft  availability  and  spares  investment  costs. 

IbmiODUCTION  TO  THE  SIMFLE  KALMAN  FILTER 

A  Kalman  filter  model  cmsists  of  two  parts:  a  state  equation  and  a 
measurement  equatkm.  The  state  equatiim  describes  the  way  the  pa¬ 
rameters  vary  over  time.  In  the  simple  modd  that  we  are  about  to 
examine,  the  mean  is  the  only  parameter  underlying  die  process  that 
will  be  of  interest  The  state  equation  of  the  simplest  Kalman  filter 
modd  is  gi'<''Bn  by 

Pt  =  Pt-i  +  wt  , 

where  is  the  mean  of  the  process  at  time  t,  is  the  value  of  the 
mf»n  at  time  t  -  1,  and  w^  is  an  erng  term  that  is  N(0,  W^),  i.e.,  nor¬ 
mally  distributed  with  mean  zero  and  variance  W^.  This  variance 
represents  the  sources  of  uncertainty  that  we  have  discussed  in  pre¬ 
ceding  paragraphs. 
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Tlw  mDatiii  fwinnr  aquatioii  rdatea  ths  men  to  the  obeervatkms  of 
the  larooefls.  The  meaeuruBADt  equatioB  of  thie  ptooeeB  is  given  by 

Yt  *  Pt  +  ▼*  . 

whare  is  N(0,  V^).  pg  has  a  nonnal  prirar  distribution  described  by 
its  mean  and  variance.  In  thiw  application,  the  prior  distributicMi  of 
the  mean  will  be  derived  from  past  obeervirtions  of  the  process  <a  of 
other,  HimilsT  inrooesses  using  an  empirical  Bayes  approach,  rather 
than  by  subjective  judgment  as  is  sometimes  the  case  in  Bayesian  ap¬ 
plications.  In  the  fidlowing  discussicm  the  caret  (^)  is  used  to  denote 
estimators  and  of  will  denote  the  specified  variance  of  p^  at  time  t. 
Usingtiiis  notation,  the  pricHr  fmr  Po  is  NiPo.Ooj .  The  distribution  of 
the  mean  will  be  updated  with  eadi  obe^vation  of  the  process. 

The  idea  is  that  we  observe  Y^,  a  realization  of  the  process  whose 
mean  is,  momentarily,  p^.  P^  is  related  to  P..1  but  is  not  forced  by  as¬ 
sumption  to  be  the  same.  T^  modriing  procedure  yields  a  natural 
approadi  to  updating  the  distribution  of  p^,  a  natural  prediction 
method  for  future  values  of  the  process,  and  prediction  error  esti¬ 
mates  that  are  more  reasonable  than  those  obtained  from  fixed  pa¬ 
rameter  modds.  The  ernn:  term,  v^  represents  stochastic  variance  as 
in  most  statistical  models.  It  affects  one  obeervation  only;  it  is  “noise” 
in  the  traditional  sense.  The  w^  is  an  error  term  of  a  diffnent  sort.  It 
is  the  difference  between  the  underlying,  nonstationary  mean  at  two 
a4jacent  points  in  time.  It  is  not  transitory;  i.e.,  it  has  a  lasting  effect 
on  the  process.  It  corresponds  to  a  variation  in  the  mean  over  time  in 
the  fashion  of  a  random  walk. 

The  computations  of  estimates  and  predictions  using  this  model  are 
straisditforward.  The  estimation  and  prediction  formulas  are  recur¬ 
sive;  the  estimate  for  p^  is  computed  firom  the  estimate  for  Pt-i  and 
from  the  current  observation. 

The  estimate  for  p^  given  =  Yj,  Y2, . . . ,  Y^  is  [63]: 

2  XMJ 

Pt  =  (1  -  a)Pt_i  +  aYt  ,  where  a  »  • 

This  can  be  thoufd^t  of  as  the  result  of  mixing  two  estimates  of  known 
variances;  p^  having  variance  of-i  ,  and  Y^  having  variance  V^. 
This  is  nearly  the  same  as  simple  exi^ential  smoothing  with  a 
smoothing  constant  of  a,  except  for  the  effect  of  the  prior,  howevor, 
the  effect  of  the  prior  diminishes  as  additional  observations  of  the 
process  are  made.  In  the  long  run,  the  predictiims  of  the  tvm  ap- 
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dal  eaaat  <d  tha  Kalman  fihar.  Mofa  aophidaeated  Kalman  fiHm 
cannot  neoeasarily  be  equated  to  aimide  exponential  amoothing. 

The  measure  of  uncertainty  (the  posterior  variance)  associated  with 
the  current  state  is  [63]: 

qf«Vta»V«  . 

Vt+o?_i+Wt 

Note  that  this  is  the  harmonic  mean  oi  the  two  variances, 
and  V^.  In  Bayesian  statistics,  this  is  the  usual  posterirar  variance 
with  a  normal  mean  when  the  observatirai  variance  is 

known. 

Several  numraical  values  are  required  to  supprart  this  method: 

Vt,  and  Wit  . 


Often,  and  are  assumed  to  be  unchanging  over  time.  They  de¬ 
scribe,  respective,  the  likely  magnitude  of  the  measurement  error 
and  the  period-to-period  variability  in  the  underiying  mean.  For 
some  applications,  it  may  be  possible  to  estimate  them  quite  accu¬ 
rately  throui^  data  analysis;  in  othra*  applications,  V  and  W  will  be 
knobs  that  need  to  be  tuned,  with  the  results  judged  by  methods  that 
we  will  discuss  below.  In  specifying  the  numeric  values  of  W  and  V, 
we  implidtfy  specify  the  ratio  of  stochastic  variance  to  the  sum  of  all 
of  the  other  sources  of  uncertainty. 


Estiiiintfaig  and  Pkndicting  tile  Updatmg  Eqnationa  fior  lliia 
Special  Case 

We  now  examine  the  procedure  for  inredieting  k  periods  into  the  fu¬ 
ture  and  for  attaching  a  measure  of  uncertainty  to  the  prediction.  Of 
course,  k  equal  to  1  is  the  most  common  case. 

Consider  the  problem  of  predicting  frar  time  t  +  k;  note  that 

Pt+k  =  Pt  +  Wt+i  +  ...+  Wj+k  and  Yj+k  =  pj+k  +  n+k  - 
so  the  prediction  is 

E(Yt.kl  Yt)  »  E(Pt  + 1 +  vt.kl  Yt)  =  Pt  . 


47 


Note  tlist  thtfe  is  no  dotenmnutic  directkii  in  Uie  variation  of  tibe 
underijrtef  mean  mnee  tiia  wpocted  vahie  of  ia  xoro;  tbna  the  ea- 
pected  value  of  the  peoceea  k  perteda  in  the  fiitiTO  ia  the  tame  aa  the 
emnrent  oatimato  of  the  mean,  hot  the  measure  of  unocvtainty  (pre¬ 
dictive  varianoe)  grows  with  ^  length  of  the  forecasting  horison  as 
shown  by 


var(Yt^klY»)«Vt  +  o?+kWt  . 


The  three  terms  in  var(Yt4.i^lYt)  correspond,  respectivdy,  to  mea¬ 
surement  error  in  Y^^^  (stodiastic  varianoe),  uncertainty  about  the 
current  value  of  ^  (parameter  variance),  and  the  variability  in  ^  ovor 
the  k  time  periods  between  t  and  t  -f  k  (variance  from  all  othor 
souremi).  Intuitiv^,  at  least,  this  logic  is  superior  to  traditional 
methods  (rf*  prediction.  We  know  that  ifwe  are  predicting  fiur  into  the 
ftituie  we  are  less  oertam  oi  our  predictions.  Tliis  logic  r^ects  that 
uncertainty. 

The  expressions  for  reflect  that  1b  really  most  affected 

hy  the  ratio  of  W  to  V,  and  that  of  is  more  affected  by  the  magnitudes 
of  V  and  W.  The  same  is  true  for  the  predictions  and  the  predictive 
varianoe:  The  prediction  is  mainly  influenced  by  the  ratio  of  W  to  V, 
the  predictive  varianoe  more  by  the  magnitudes  of  the  variances. 


Building  Intuition  About  the  Simide  Kalman  Filter  Model 

The  Kalman  filters  that  seem  most  appropriate  for  spare  parts  de¬ 
mand  forecasting  are  somewhat  more  elaborate  than  this  simple 
model.  Nevertheless,  building  some  inttiitimi  about  this  simple  model 
may  be  useful  in  understanding  the  strengths  and  weaknesses  of  this 
forecasting  approach.  We  will  examine  four  archetypal  data  series 
and  observe  how  the  Kalman  filtm*  forecasts  each  one;  step  function, 
outlier,  ramp,  and  osdllaticm.  For  each  of  these  data  series  we  show 
the  Kalman  filter’s  predictions  one  period  ahead,  in  some  cases  with 
several  different  ratios  of  state  variance  to  measurement  variance. 
We  win  contrast  these  predictions  to  those  from  the  traditional  ei^t- 
quarter  moving  average.  For  all  of  these  illustrations,  the  priors  for 
the  Kalman  filters  were  set  to  a  mean  of  0  and  a  variance  of  1.  The 
effect  of  changing  this  prior  is  discussed  below.  As  mentioned  above, 
the  predictions  of  a  Kalman  filter  are  primarily  affected  1^  the  ratio 
of  the  state  variance  to  the  measurement  variance.  This  ratio  is  in¬ 
cluded  in  the  labels  for  the  predictions  used  here;  e.g.,  “Kalman  1:4” 
implies  that  the  measurement  variance  is  four  times  as  large  as  the 
state  variance. 
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Figure  4.1  shows  the  response  of  several  Kalman  filters  and  an  eight- 
quarter  moving  average  to  a  change  in  level  (step  fimction)  in  the 
data.  Note  that  the  higher  the  ratio  of  state  variance  to  measurement 
variance,  the  more  quickly  the  predictions  adjust,  because  a  high 
value  of  this  ratio  more  heavily  favors  the  observed  data.  Part  of  the 
art  of  Kalman  filtering  is  adjusting  this  ratio  to  respond  to  real 
changes  in  the  mean  without  making  it  so  sensitive  that  it  leaps 
around  in  response  to  noise  in  the  data. 

Figure  4.2  illustrates  the  response  of  a  Kalman  filter  to  a  spike  in  the 
data.  The  prediction  responds  to  the  large  data  value  but  then  dies 
down  fairly  quickly.  The  moving  average  does  not  react  as  much  but 
the  reaction  persists  for  a  longer  period. 

Figure  4.3  shows  the  response  of  the  Kalman  filter  to  a  ramp  in  the 
process.  Note  that  the  Kalman  filter’s  predictions  are  closer  to  the  ac¬ 
tual  data  than  the  predictions  of  the  eight-quarter  moving  average  in 
all  but  one  period.  The  Kalman  filter  tracks  the  ramp  better  than  the 
moving  average  and  then  responds  quickly  to  the  return  to  zero. 

Figure  4.4  illustrates  the  response  of  the  Kalman  filter  to  an  oscillat¬ 
ing  process.  This  is  the  kind  of  process  that  might  result  fi*om  annual 
cycles  in  the  data.  This  example  illustrates  when  a  simple  Kalman 
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F^ure  4.1 — A  Step  Function 
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Figure  4.4 — An  Oscillating  Process 

filter  mi^t  be  inappropriate.  More  elaborate  Kalman  filters  have 
been  designed  to  deal  with  oydic  behavior.  If  a  particular  Kalman 
filter  model  forecasts  poorly,  there  are  many  extensions  and 
enhancements  available  to  treat  most  common  forecasting  problems. 

Figure  4.5  addresses  the  effect  of  the  prior  on  the  Kalman  filter’s 
predictions.  This  graph  changes  the  prior  mean  to  5.0  and  leaves  all 
other  values  unchani^  The  effect  of  the  prior  has  disappeared  by 
the  eighth  period.  Since  we  are  comparing  the  forecasts  to  those  firom 
an  eight-quarter  moving  average  that  doesn’t  start  producing  fore¬ 
casts  until  period  nine,  the  prior  has  no  effect  here.  In  general,  the 
prior  could  have  an  effect  afto  m^t  periods  if  the  state  variance  and 
the  prior  variance  were  veiy  nnall  or  if  the  prior  mean  were  very 
large.  In  later  applications  in  this  report  we  will  use  noninformative 
priors  and  will  be  careful  to  avoid  hif^y  influmitial  (informative)  pri¬ 
ors. 


Figure  4Ji— -Hie  Effect  of  the  Prior  on  the  Forecajrte 


I  I  INTRODUCTION  TO  THE  GENERAL  KALMAN  FILTER 

!  MODEL 

i 

!  I  A  general  Kalman  filter  model  allows  us  to  handle  more  realistic 

measurement  equations  and  state  equations.  Explanatory  variaUes 
>  may  be  added  to  the  measurement  equation  and  trmuls  or  cycles  may 

be  added  to  the  state  equation.  The  state  equation  can  allow  different 
I  rates  of  change  for  different  parameters  of  the  model.  These  features 

i  I  make  the  general  model  especially  flexible  and  powerful  in  modeling 

i  I  more  complicated  processes. 


Developmoit  of  the  General  Kalman  Filter  Model 

The  Kalman  filter  model  can  be  generalized  to  vector-valued  paarame- 
ters  and  multivariate  observations.  The  formulas  are  quite  similar, 
except  that  vectors  and  maiaices  replace  the  scalar  values.  The  state 
equation  of  the  general  modd  is  given  by 


p,  =  A, .  Pt_j  +  w 


t  ' 


and  the  measurement  equation  by 
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Yt  *  Xj  •  Pt  +  Vt  , 

when  is  distributed  N(0,  W^)  and  is  distributed  N(0,  V^),  whme 
tlie  N  dniotes  tlie  iMnrmal  distributicHi.  Pg  has  a  mamal  {nirar  de¬ 
scribed  by  its  mean  and  variance.  P|  is  a  adunm  vector,  Af  is  a  square 
matrix,  and  is  a  matrix.  Note  bIm  that  need  not  be  square  and 
and  P^  need  not  be  of  the  same  length.  Indeed  Y^  is  often  a  scalar, 
and  are  now  covariance  matrices  rather  than  scalar  variances. 
We  use  this  "normal-normal*  model  for  computational  convenience. 
Below  we  discuss  the  use  d  transformations  to  fit  the  discrete  nature 
of  the  data  better. 

This  more  general  model  will  be  used  in  the  demand  forecasting  ex- 
amide  given  bdow  and  will  be  djeaissed  as  it  applies  to  demand  fore¬ 
casting  in  following  sections. 

Building  Intuition  About  the  General  Model 

It  may  be  helpful  to  examine  an  application  of  this  general  model  to  a 
specific  example.  The  following  example  uses  data  on  a  component  of 
the  F-111  aircraft,  an  airspeed  indicator. 

In  this  application,  the  number  of  observed  demands  in  a  time  period 
(quarter)  is  modeled  as 


“  Pot  Pit^t  +  • 

given  Pq(  and  p^,  where  ^  denotes  hundreds  of  hours  fiown  by  aircraft 
in  which  the  airspeed  in^cator  is  installed;  i.e.,  ^  =  20  means  that  the 
airspeed  indicator  experienced  2,000  hours  of  use.  This  is  the 
measurement  equation.  The  error  term  v^  is  normal  with  variance 
to  be  specified.  The  quarter-to-quarter  evolution  d  p^  will  be  modeled 
by  the  state  equation 


Pt+i  ”  Pt  » 

where  w^  is  a  bi-variate  normal  with  zero  covariance  and  variances 
and  Wjt  to  be  specified.  P^is  the  vector  containing  Pq^  and  P^^. 
Thus,  in  tiie  notation  of  the  previous  example, 

Pt  =  (Pot  >  Pit )  treated  as  a  column  vector. 


Ft  =  [l,ftl  treated asalx  2matrix. 
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Some  need  to  be  specified  to  start  up  tbe  Kalman  filter  ma- 

diinety.  In  the  current  i»oblem,  these  iwmbers  are 

•  The  mean  and  covariance  matrix  for  ^  two  means,  two  variances, 
and  a  covariance, 

■  The  two  state  variances  W|t  and  (the  state  covariance  is  speci¬ 
fied  to  be  zero  here),  and 

*  The  measurement  variance 

This  case  differs  firom  the  simple  example  in  that,  in  the  former  case, 
the  state  and  the  measurement  were  on  the  same  scale.  In  this  case, 
the  state  is  described  by  a  vector  of  regression  coefficients,  and  the 
measurement  is  an  observed  number  of  demands.  They  are  connected 
by  F^.  One  implicati<m  of  this  is  that  the  state  and  measurement  vari¬ 
ances  are  on  different  scales,  which  makes  them  somewhat  harder  to 
compare.  The  state  inropagates  by  an  identity  matrix;  i.e.,  the  only 
change  from  one  period  to  the  next  is  stochastic. 

In  this  discussion,  we  will  use  a  somewhat  ad  hoc  method  of  setting 
the  starting  values  for  the  filter.  For  the  prior  values  we  will  just 
regress  the  first  eight  quarters  of  demands  on  the  flying  hours  and 
use  the  parameter  estimates  firom  this  regression  for  our  prior  values 
for  Pq.  We  will  use  the  estimated  covariance  matrix  fi*om  this  regres¬ 
sion  for  o§.  Setting  the  state  and  measurement  variances  vnll  be  a 
two-stage  procedure.  First,  we  will  divide  the  variance  of  the  first  , 
eight  quarters  of  demand  and  explore  alternative  allocations  of  that 
variance  between  the  state  and  measurement  equations.  Second,  in 
the  state  equation  we  will  divide  the  variance  between  Pit 

equally.  Half  of  the  state  variance  will  be  assigned  to  ^ot !  other 
half  will  be  divided  by  the  mean  of  the  first  ei^t  quarters’  flying 
hours  and  assigned  to  This  division  will  make  the  cmitribution 
firom  the  variance  of  Pit  (alter  it  has  been  multiplied  by  flying  hours) 
to  the  overall  variance  rou^y  equal  to  the  contribution  firom  Pot  • 

A  full  Bayesian  treatment  of  this  problem  would  require  much  more 
careful  thinking  about  the  prior  values  for  each  part.  Since  it  is  ulti¬ 
mately  impractical  for  us  to  engage  in  a  careful  specification  of  the 
prior  for  each  of  the  parts  in  the  Air  Force  inventory  system,  we  must 
use  simpler  rules.  We  can  evaluate  different  rules  for  specifying 
the  priors  by  observing  the  quality  of  their  predictions  over  a  wide 
range  of  parts.  This  is  an  ezanq>le  of  a  pragmatic  application  of  the 
Bayesian  paradigm.  In  a  discussion  helow,  we  will  use  a  more  easily 
implemented  approximation  to  th  iman  filter,  the  weighted  re- 
greeeion  forecaster. 
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FiguTM  4.6  and  4.7  ilhuainate  Ute  efiaets  of  the  variances  on  the  pro- 
dictions  of  thk  ^ype  of  modri.  Figure  4.6  shows  the  r^ilts  of  making 
the  state  and  measoronent  variances  rou^y  equal.  Note  that  the 
pradktions  gmeraQy  increase  witih  the  observed  data  until  the  middle 
oi  the  samide  period  and  then  drop  (town.  Generally,  the  pattern  is 
that  predictions  lag  the  demands  by  one  period;  i.e.,  if  there  is  a  drop 
in  demands,  the  predictions  typically  dnq)  in  the  fcdlowing  poriod. 
This  is  not  necessarily  the  case,  however;  demands  increased  slisd^tly 
in  period  26  but  the  fuedictiim  decreased  in  period  27  owing  to  a 
change  in  flying  h<Mirs. 

Figure  4.7  illustrates  what  haiq>en8  if  the  state  variance  is  set  to  zero, 
oorresp(B)ding  to  fitting  a  r^ression  to  all  the  data  up  to  time  t  unH 
then  using  the  regressum  to  predict  for  time  t  +  1.  Notice  that  in  the 
mkhlle  of  the  sample  pmiod  the  predictions  undmpredict  demand. 
This  ha|q>ens  because  this  model  is  still  giving  foil  wei^t  in  its  pre- 
chctions  to  the  eariy  periods,  but  the  world  has  evolved  away  from  the 
demand  rate  that  described  this  eariy  period. 

Values  of  state  and  measurement  variances  that  are  between  these 
two  cases  w<Hild  provide  intermediate  levels  of  smoothing.  Note  that 
making  the  state  variance  large  will  not  produce  a  flat  horizontal  line; 
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Figure  4.6 — State  Variance  and  Measatanent  Variance  Equal 


4  6  8  10  12  14  16  18  2022242626303234363840 

Quartan 

Figure  4.7 — State  Variance  Set  to  Zero 


i.e.,  it  smooths  the  imderl3nng  regression  parameters,  not  the  de¬ 
mands.  The  effects  of  the  variation  in  flying  hours  will  still  be  seen  in 
this  graph. 


Robastiieaa  Kalaum  Filtera 

Why  is  it  advantageous  to  use  time-varying  coefficients  instead  of  a 
bigger  variance  on  the  error  term  v^?  Suppose  some  function  relates 
expected  demands  to  flying  hours. 

•  If  the  function  is  linear  and  unchanging  in  time,  then  the  Kalman 
filter  has  to  do  worse  than  a  model  that  assumes  fixed  coefficients, 
because  the  Kalman  filter  makes  less  use  of  older  observations 
when  they  are  just  as  good  as  recent  ones.  An  example  would  be 
using  the  sample  mean  as  opposed  to  exponential  smoothing  to  es¬ 
timate  a  stationary  population  mean,  a  case  in  which  exponential 
smoothing  is  inefficient. 

■  If  the  function  doesn’t  vary  in  time  and  is  nonlinear,  the  Kalman 
filter  will  be  better  than  a  fixed  parameter  model  if  the  flying  hours 
are  positively  autocorrelated  in  time,  because  at  any  given  time  the 
Kalman  filt^  is  linearly  apimndmating  the  con^  part  of  the 
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ciirv»~-tluit  is,  the  pert  id  the  curve  essodated  with  the  current  fly¬ 
ing  hours— because  the  iweaumed  antootnrdatioa  of  Qsring  hours. 

•  If,  (m  the  otiier  hand,  the  oirve  reaUy  is  linear  but  the  sl<^  and 
interoq)t  vaiy  over  time  (for  vriiatever  reason,  sudi  as  omitted 
variddes),  the  Kalman  filter  woito  better  than  t^  fixed  parameter 
modd  if  the  slope  and  intoroept  are  positivdy  autoonrirelated  in 
time:  (Md  data  tell  you  less  about  whece  the  line  is  than  do  currmit 
data.  Again,  if  the  mean  is  a  random  walk,  the  expcmmtial 
nnoother  will  beat  the  samide  mean  in  estimating  the  cuirmt 
value  if  the  variance  of  the  randmn  walk  (W^)  is  large  enough. 

In  general,  we  may  observe  b(dh  ncmlinearity  in  the  relationship  be- 
twem  flying  hours  aid  eiqiected  demands  and  a  changing  rdation- 
ship  over  time.  Ihe  Kalman  filter,  in  effect,  smooths  in  both  time  and 
in  the  regressors,  thus  achieving  greater  adaptability  in  the  face  of 
changing  parameters  and  flying  hour  programs.  If  the  required  auto- 
ocarelations  are  present,  the  Kalman  filter  should  do  better.  This 
may  help  explain  the  observed  (but  somewhat  mysterious)  robustness 
of  tile  Kalman  filter. 

Relationship  to  Othm-  Prediction  Methods 

One  useful  characteristic  of  Kalman  filter  models  is  the  way  they 
generalize  other  well  known  time  series  and  forecasting  methods. 
Unfortunately,  this  has  caused  some  confiision  in  the  literature.  It  is 
not  correct  to  claim  that  Kalman  filtering  is  ‘^just  exponential 
smoothing”  as  is  sometimes  heard.  Exponential  smoothing  is  a  spe¬ 
cial  case  of  Kalman  filtering  but  Kalman  filtering  incorporates  a 
much  richer  collection  of  models.  In  Appendix  A,  we  discuss  some 
well  known  statistical  models  and  how  they  can  be  vnritten  as  Kalman 
filter  models. 


5.  MORE  n4EXIBLE  DEMAND  AND  NBTS 
FORECASTS 


In  this  aectiim  we  will  two  alternative  forecaating  metiiods  to 
the  demanda  fiar  a  aample  of  aircraft  qMie  parts.  Botin  of  these  meth¬ 
od  are  motivated  Iqr  a  need  fior  greater  fleadnlity  to  oope  witii  the 
nonstationarity  of  the  demand  process.  The  two  basic  types  of  predie- 
tion  methods  we  consider  here  are  (1)  weifdd^  calculation  of  demand 
rates,  and  (2)  wei|d>ted  regresskm  demand  forecasters.  We  also 
examined  the  performance  of  Kalman  filter  regression  finecaaters  in 
this  research  but  concluded  tinat  the  wei^ted  r^ressioin  methods 
performed  at  least  as  well  and  were  more  easier  implemented.  The 
weiid^ted  r^resdon  forecaster  can  be  thoug  ht  of  as  a  special  limiting 
case  of  the  Kalman  filter  r^ressimi  forecastor  that  ineorpmntes  a 
noninformative  prior;  therefore,  it  has  the  qiecial  advantage  in  an 
application  as  large  as  the  Air  Force’s  reooveraUe  spares  inventory 
system  of  not  requiring  the  spedficatiim  of  the  priors  needed  for  the 
Kalman  filter  repression  forecaster  [63]. 

The  quality  of  predictions  is  evaluated  here  in  two  difierent  ways, 
root  mean  squared  error  (RMSE)  and  mean  absolute  deviation 
(MAD).  In  subsequent  discussion,  we  evaluate  the  predictions  in  ac¬ 
tual  application  to  spares  requiranents  computations  with  a  capabil¬ 
ity  assessment  mo^l,  Byna>M£TKIC  Version  6.  Root  mean  stpiared 
error  is  defined  as 


RMSE 


■S 


[  back-transfonnedpiedicted  value^  -  Dj  ) 
n 


where  D{  is  the  demand  in  period  i  and  the  bac^-transformed 
predicted  value  is  typically  the  prediction  fium  the  model.  In  a  few  of 
the  models  consideied  here,  the  forecast  variable  is  in  tiie  square  root 
scale,  so  back-transforming  (in  this  case  squaring)  is  required  to  put 
the  predictions  in  the  apiaopriate  scale. 

Mean  absolute  deviation  is  defined  as 


A  |back-transf6rnied  predicted  value)  -  Dj 


i=i 


where  the  I  I  symbol  stands  for  the  absolute  value.  This  summary  of 
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the  quality  of  the  predietMoe  weighte  huger  predictive  errors  lees 
sevordy  than  RMSE  <k>es.  Whether  tiiese  or  any  other  measures  of 
predictive  quality  are  aKtrojariate  depends  on  the  application,  but 
these  measures  will  do  fisr  illustrative  purposes. 

Throughout  tibis  section  we  will  use  these  measures  of  predictive 
quality  to  compare  alternative  fiuecasting  methods  with  the  forecast¬ 
ing  method  bmeed  <m  the  ei^t-quarter  moving  average  now  used  by 
theAir  Fcnm 

ffince  our  motivation  for  exploring  alternative  modding  aid 

forecasting  techniques  is  to  improve  the  estimation  of  requiremmits 
for  aircraft  recoverable  spares  and  d^>ot-levd  repair,  we  are  inter¬ 
ested  in  the  peifttrmance  of  alternative  forecasting  techniques  over 
the  time  horizons  sqiplicaUe  to  the  requirmnents  estimation  problem. 
Fcr  depot-levd  repair,  as  a  practical  matter,  the  fmecasting  horizon  is 
<mly  two  quarters.  Altiiou|^  repair  planning  is  also  done  over  longer 
planning  horizons,  the  Air  Logistics  Center  has  considerable  quarter- 
to-quarter  flexibility  in  a4iusting  repair  quantities.  For  spares  pro¬ 
curement  purposes,  however,  planning  horizons  are  typicdly  about 
three  years  long  because  of  the  source  sdection  and  manufacturing 
IHOcesses  invdved.  Exploration  of  methods  that  mi^t  help  alleviate 
those  constraints  are  beyond  the  scope  of  this  work.  Therefore,  we 
wUl  deal  with  these  longer  horizons  in  the  requirements  forecasting 
problem  and  will  examine  the  perfmrmance  of  dtemative  forecasting 
methods  over  planning  horizms  as  long  as  13  quarters. 

TBE  PARTS  SAMPLE 

The  sample  of  data  used  in  these  muunples  consists  of  BP15  recover¬ 
able  aircraft  spares  from  D041  with  one  or  more  demands  in  the  pre¬ 
vious  ei{^t  quarters  and  a  program  select  code  of  one  (i.e.,  the  de¬ 
mands  f<Hr  these  parts  are  assumed  to  be  driven  by  flying  hours).  The 
time  period  of  the  data  runs  from  March  1980  to  March  1989  by  quar¬ 
ters.  The  sample  was  further  restricted  to  parts  that  had  incurred 
mme  than  four  demands  systemwide  in  at  least  one  of  the  40  quar¬ 
ters.  There  is  little  reason  to  expect  that  the  techniques  we  consider 
here  can  improve  demand  predictions  for  parts  that  never  generated 
mote  than  four  demands  in  a  quarts  over  40  quarters.  We  also  re¬ 
stricted  the  sample  to  parts  that  had  valid  demands  (zero  is  a  valid 
demand;  missing  is  not)  in  at  least  12  quarters.  Although  the  predic¬ 
tion  tedmiques  considered  here  need  at  most  eig^t  quarters  of  data  to 
get  started,  we  need  a  few  quarters  of  data  with  which  to  evaluate  the 
quality  their  inredictions. 
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Miwnng  valiiea  and  zero  demanda  are  iasuaa  in  these  data.  We  use  an 
(^>arati<Hial  definitUm:  If  there  are  Qying  hours  in  the  reaml  fw  the 
part  and  zero  or  missing  demands,  thm  we  infisr  that  the  part  had 
zero  demands.  If  the  pert  has  zero  dmnands  and  the  flying  hcnurs  are 
missing,  then  the  demands  are  missing. 

These  restrictions  left  us  with  14,007  parts  in  our  universe.  We 
stratified  these  parts  into  three  groups.  Group  1  consisted  oi  parts 
with  an  average  demand  ci  more  than  15  per  quarter  in  the  quarters 
where  the  demands  woe  not  missing  fiom  the  database  (4,216  parts). 
Groups  2  and  3  both  had  less  than  15  demands  per  quaiW  cm  aver¬ 
age.  Group  2  consisted  of  parts  that  cost  less  than  |2,5(H)  (4,966 
parts).  Group  3  consisted  of  parts  that  cost  more  than  $2,500  (4,826 
parts).  Frmn  each  of  these  thm  groups  we  selected  a  random  sample 
of 200  parts  tor  the  evaluations. 

METHCMMS  AND  KESULTS 

In  the  discussion  that  follows,  we  explore  the  performance  of 
weii^ted  calculation  of  dmnand  rates  and  wei^ted  regression  fore¬ 
casters. 


Wei|d>ted  Calculation  of  Demand  Rates 

A  simple  compromise  between  the  more  sophisticated  forecasting 
methods  considered  in  this  section  and  the  Air  Force’s  current  ei^t- 
quarter  moving  average  forecasts  is  wei^ted  demand  rates.  The  cur¬ 
rent  system  calculates  the  demand  rate  as 

ID, 

Demand  rate^^k  =  —  » 

Ifl»i 

i=t-7 

where  D,  denotes  the  ith  quarter’s  demands,  fh,  the  fl3dng  hours  in 
the  ith  quarter,  and  k  the  number  of  quart^  into  the  future. 
Forecasts  are  then  made  by  simply  multipl3ring  this  rate  times  the  fu¬ 
ture  program: 

Predicted  demands^^^  =  demand  ratet+k  fht.,.]^  . 


A  simple  way  to  make  this  model  more  responsive  is  to  weight  recent 
observations  more  heavily  than  older  observations: 


Demand  ratet^k 


t 


l-t-7 

A  Kalman  filtering  approach  suggests  wei|^ts  of  the  form  Wj  -  . 

We  tried  values  of  a  of  0.26,  0.5,  and  0.75;  the  results  for  a  =  0.25 
were  aberrant  and  are  not  included  here.  We  also  tried  a  four-quarter 
moving  average  without  wei^ta  aa  another  ad  hoc  way  to  gain  some 
re8p(m8ivenees  in  the  forecasts.  Figures  5.1  and  5.2  reflect  the  roc^ 
mean  squared  error  and  mean  absolute  deviation,  respectively,  over 
various  time  horizons  up  to  13  quarters  in  length,  for  the  four  ap¬ 
proaches.  These  are  calculated  firom  the  sums  of  tlm  predictive  qual¬ 
ity  measures  over  all  the  parts  in  Group  1.  We  tried  these  methods 
for  Groups  2  and  3  as  weU,  but  they  did  not  perform  better  than  the 
current  approtudi,  possibly  becatise  of  the  failure  of  the  normality  as¬ 
sumption  or  insufficient  information  content  in  low-demand  data. 
These  results  show  that  weq^ted  demand  rate  forecasters  perform 
better  than  does  the  current  approach.  The  fact  that  the  four-quarter 
moving  average  also  does  better  than  the  current  approach  suggests 
that  the  current  approach  is  not  sufficiently  responsive  to  non- 
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Figure  5.3 — ^Root  Mean  Squared  Error  of 
Weighted  Regression  Forecasters 


of  a.  However,  there  are  additional  gains  to  be  made  by  adjusting  the 
value  of  a  for  both  approaches. 

THE  COMPUTATIONAL  ALGEBRA 

The  weighted  regressio  brecaster  we  recommend  is  specified  in  the 
following  algebra.  Suppose  we  observe  the  most  recent  eight  quarters 
of  demands  and  past  item  flying  hours  in  the  sequence  dj,  d2, . . . ,  dg, 
and  fhi,  fhj, . . . ,  fhg,  respectively.  We  will  use  the  weighting  factors 
w^,  W2 . and  Wg  to  assign  greater  weight  to  the  more  recent  quar¬ 

ters  by  setting  the  {wj )  equal  to  0. 75®"' ,  1  =  1,  2, . . . ,  8  .  Thus  the 
wei^ts  will  be 

wi  =  0. 75^  =  0. 1335,  Wj  =  0. 75®  =  0. 1780,  Wg  =  0. 2373, 

W4  =  0. 3164,  Wg  =  0. 4219,  Wg  =  0. 5625,  W7  =  0. 75^  =  0. 75,  and 
Wg  =  0. 75®  =  1. 0.  The  sum  of  {Wj  I  =  3. 5995 . 
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Figure  5.4— Mean  Absolute  Deviation  of  Weighted 
Regression  Forecasters 


Let  the  notation  Zj  be  defined  to  mean  the  sum  taken  over  the  ei^t 
quarters.  The  wei^ted  mean  demand,  D  *  ,  is  given  by 

D*  =  XiMi/S,"!  . 

and  the  weighted  mean  flying  hours  (item  program),  fli*,  by 

We  define 

1^^=  (Xi^i)  (XjWidifhi)  -  (£,Widi)  (SljWifhi)  , 

Sx*  =  ]-(£,w,£h,f  . 


and 
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Then 

If  Pi  is  negative,  revert  to  the  ei|^t-quarter  moving  average. 

NRTS  FORECASTING 

NBTS  forecasting  is  especially  important  in  depot  repair  planning. 
NRTS  actions  are  one  siibset  of  base-level  demands,  the  other  two  be¬ 
ing  repairs  and  condemnations.  Over  longer  planning  horizons,  the 
current  methods  are  subject  to  large  errors.  We  explored  alternatives 
to  the  current  method  and  describe  the  results  here. 

An  improved  method  of  foi^casting  NRTS  actions  results  from  com¬ 
bining  the  improved  demand  forecasting  method  vdth  an  improved 
method  of  forecasting  the  NRTS  rate.  Again,  the  method  is  recom¬ 
mended  for  hi^-demand  items.  The  forecast  of  the  NRTS  rate  is 
done  by  weighting  more  recent  observations  of  the  NRTS  rate  more 
heavily  than  older  observations  using  a  simple  exponential  smoothing 
technique  and  a  weightir^  factor  of  0.75.  In  the  notation  just  intro¬ 
duced, 

NRTS  rate  =  2].WiNRTSi/5^.Widi, 

where  NRTS|  is  the  munber  of  NRTS  actions  in  quarter  i. 

This  NRTS  rate  estimator,  combined  with  the  improved  demand  fore¬ 
casting  technique  already  discussed,  delivers  ^e  performance  re¬ 
flected  in  Figures  5.5  and  5.6  for  hig^-demand  items.  The  alternative 
method  dominates  the  current  method  in  root  mean  squared  error 
over  all  planning  horizons  examined.  In  mean  absolute  deviation, 
though,  the  current  method  does  sli^tly  better  for  shorter  planning 
horizons.  The  alternative  method  is  our  choice  for  any  planning  hori¬ 
zon. 


CONCLUSIONS 

The  compelling  message  of  Figures  5.3  and  5.4  is  that  any  one  of 
these  demand  forecasting  methods  clearly  dominates  the  eight-quar¬ 
ter  moving  aven^  and  their  superiority  increases  with  the  length  of 
the  forecasting  horizon.  The  question  remains,  then,  of  which  method 
to  choose.  For  two  reasons,  our  choice  is  the  wei^ted  regression 
forecaster  with  an  a  of  0.75:  (a)  It  scored  best  in  its  average  ranking 
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Figure  6.5— Root  Mean  Squared  Error  of  Improved  NRTS  Forecaater 
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Figure  5.6— Mean  Absolute  Deviation  of  Improved  NRTS  Forecaster 


among  RMSE  and  MAD  scores  relative  to  the  other  methods,  and  (b) 
it  is  more  intuitively  appealing  than  methods  that  effectively  use  any 
less  of  the  available  data.  For  example,  the  four-quarter  moving  av¬ 
erage  did  quite  weU  in  terms  of  mean  absolute  deviation  but  depends 
on  four  quarterly  observations  to  estimate  two  parameters,  hardly  an 
appealing  idea.  It  is  also  appealing  that  the  weighted  regression  fore- 
ca^r  with  an  a  of  0.75  does  not  have  such  a  steep  slope  in  its 


weighting  ot  the  ei^t  quarters  of  observations  as  do  weii^ted 
regression  methods  with  smallm  values  of  ot. 

It  is  impmtant  to  understand  that  tiie  apparent  superiority  of  the 
wei^ted  regression  demand  fiaecasters  over  the  wei^^ted  demand 
rate  forecasters  partially  derives  from  the  inclusion  of  a  non-zero  in¬ 
tercept  in  the  model  of  demands  as  a  function  of  flying  hours.  This 
feature  is  important;  it  departs  from  an  assumption  of  strict  propor¬ 
tionality  between  demands  and  flying  hours.  The  weighted  regres¬ 
sion  demand  forecasting  procedure  explicitly  estimates  the  intercept. 

By  both  of  our  evaluative  critmia,  the  a  =  0.75  weighted  regression 
demand  forecaster  does  fairly  well  for  Group  1.  Therefore,  for  items 
in  Group  1,  we  recmnmend  it  as  the  default  demand  forecaster  for 
spares  requirements  computations.  In  any  event,  the  use  of  the  eight- 
quarter  moving  average  should  be  discontinued  for  high-demand 
items. 

For  NRTS  forecasting,  the  improved  NRTS  forecaster  incorporates 
the  improved  demand  forecaster  and  adds  an  improved  NRTS  rate 
forecaster  that  also  uses  a  0.75  wei^ting  factor.  It  dominates  the 
current  method  in  root  mean  squared  error  over  all  planning  horizons 
examined  and  dominates  the  current  method  in  mean  absolute  devia¬ 
tion  over  long  planning  horizons.  We  recommend  its  use  as  the  de¬ 
fault  NRTS  forecaster  for  items  in  Group  1.  We  propose  no  change  for 
Groups  2  or  3. 


6.  VARIANCE  ESTIMATION 


Estimatuig  the  variance  of  demands  for  aircraft  reooveraMe  spare 
parts  is  difficult  for  two  reasons:  (a)  The  statistical  fmcddem  of  vari¬ 
ance  estimation  is,  in  itself,  difficult  because  estimators  of  variances 
typicaQy  have  very  error  variances  themselves,  and  (b)  because 
our  model  of  the  demand  process  is  so  imperfect,  the  effects  of  maiqr 
fectors  on  the  observed  variability  in  the  process  are  lumped  into  our 
estimate  of  the  variance,  as  we  discussed  in  Section  4.  In  this  section, 
we  discuss  some  characteristics  of  the  VTMR  estimator  introduced  in 
Section  3,  suggest  an  improvement  to  variance  estimation  that  ccn*- 
rects  a  logical  flaw  in  the  current  system’s  specification  of  variance 
and,  finally,  mention  a  few  approaches  to  variance  estimation  that  did 
not  work  well. 

SOME  OBSERVATIONS  ABOUT  THE  VTMR  ESTIMATOR 

To  understand  better  the  role  of  nonstationarity  and  data  partitioning 
in  estimating  the  underlying  VTMR,  we  explored  some  of  the  charac¬ 
teristics  of  the  VTMR  estimator  in  both  stationary  and  nonstationary 
processes:  (a)  The  variance  of  the  estimator  increases  with  the 
coarseness  of  the  partitioning  and  the  true  VTMR  even  when  the  pro¬ 
cess  is  stationary,  and  (b)  very  serious  overestimation  of  the  true 
VTMR  (i.e.,  the  VTMR  of  demands  around  the  mean,  stationary  or 
nonstationary)  occurs  in  the  presence  of  nonstationarity,  and  the 
overestimation  is  an  increasing  function  of  the  demand  rate.  These 
findings  are  fundamentally  important  to  any  use  of  the  VTMR  esti¬ 
mator  in  spares  and  repair  requirements  computations.  They  are 
comnstent  with  (a)  the  association  of  larger  values  of  observed  VTMRs 
with  hi^-demand  items,  (b)  the  substantial  differences  between  the 
distributions  of  observed  VTMRs  discussed  in  the  context  of  the 
Bitburg  data  analysis  and  those  discussed  by  Crawford  [2],  and  (c)  the 
wildness  of  the  ffistribution  of  observed  VTMRs  around  the  power 
function  used  by  AFMC  to  specify  VTMRs  to  describe  the  probability 
distribiT'’^.on  of  the  number  of  items  each  type  in  resupply.  Forecast 
variance  is  discussed  below. 

Consider  a  simple  Poisson  demand  process  with  a  mean  of  10  per 
quarter.  Given  this  stationary  process  and  eight  quarters  of  observa¬ 
tions,  the  VTMR  estimator  has  a  standard  deviation  of  0.53.  Even 
with  this  very  well  behaved  process,  we  can  mq>ect  to  observe  a 
VTMR  of  about  1.9  nc  more  about  five  times  out  of  100.  Thus,  in  ob- 
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serving  a  substantial  number  of  parts  demand  histories,  one  should 
not  be  surprised  to  see  some  large  VTMRs,  even  if  the  underlying  pro¬ 
cesses  are  simple  PoissoiL  It  may  be  incorrect  to  condude  that  some 
of  the  {MTocesses  have  large  tmderijnng  VTMRs.  The  risk  here  is  that 
we  may  tend  to  confuse  the  large  variability  in  the  sampling  distribu¬ 
tion  of  the  VTMR  estimator  with  variability  in  the  underl3ring  process. 

The  same  risk  applies  to  inferences  in  the  other  direction.  We  may 
observe  a  low  VTMR  for  any  individual  part  when,  in  fact,  the  VTMR 
underiying  its  demand  process  is  substantially  greater  than  the  value 
of  the  estimator.  One  i^proadi  to  reducing  our  vulnerability  to  error 
in  estimating  the  variaWity  undertying  a  demand  process  for  an  in¬ 
dividual  part  is  to  pool  our  estimates  of  variance  across  parts.  In  the 
current  system,  this  pooling  is  accomplished  throu^  the  application 
of  the  power  function  discussed  previously.  It  may  seem  unsatisfying, 
but  it  does  eliminate  much  of  the  instability  in  variance  estimation 
across  time  that  would  occur  if  variances  were  estimated  part  by  part. 
Part-specific  variance  estimation  would  also  induce  much  greater 
volatility  in  numerical  values  of  the  AFAO  over  time,  in  turn  inducing 
inrocurement  actions  in  response  to  momentary  changes  in  the  esti¬ 
mated  variances  of  specific  parts. 

Effects  of  Bias,  Data  Partitioiiing,  and  Nonstationarity 
on  file  VTMR  Estimator 

We  describe  briefly  in  the  paragraphs  that  follow  the  bias  function  of 
the  VTMR  estimator  and  its  mean  and  standard  deviation  as  a  func¬ 
tion  of  data  partitioning,  mean  demand  rate,  underlying  VTMR,  and 
two  specific  examples  of  nonstationarity  in  demand.  The  results  are 
important  in  their  implications  for  spares  and  repair  requirements 
estimation  as  well  as  in  explaining  some  of  the  observations  of  earlier 
researchers  in  this  area.  They  also  serve  to  underscore  the  difficulty 
of  the  VTMR  estimation  problem. 

In  Figures  6.1  through  6.3,  we  hold  the  total  time  period  constant  and 
subdivide  it  into  periods  of  various  loogths.  The  mean  we  refer  to  is 
the  mean  number  of  demands  per  time  period. 

Figure  6.1  reflects  the  VTMR  estimator’s  bias  function  with  demand 
processes  with  underlying  VTMRs  of  1.0,  2.0,  and  3.0.  With  a  sta¬ 
tionary  process,  the  expected  value  of  the  VTMR  estimator  is  unaf¬ 
fected  by  the  partitioning  of  the  data.  The  bias  is  very  modest  above 
50  or  so  total  demands  in  the  time  pmiod.  Since  tlm  hty^-demand 
parts  tend  to  drive  the  performance  of  the  inventory  system,  the  ef¬ 
fects  of  the  bias  function  are  probably  not  important  in  shaping  sys- 
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FiKure  6.1 — ^Effect  of  the  Estimator'*  Bias  Function 
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Figure  6.2— Effect  of  Data  Partitioning  on  the  Standard 
Deviation  of  the  VTMR  Estimator 
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Figure  6.3 — Effect  of  Data  Pertitioiiiiig  on  the 
Expected  Value  of  the  VmR  Estfaiator 


tern  perfcumance.  At  the  systemwide  level  used  by  AFMC  to  compute 
spares  requirements,  the  bias  function  does  not  play  nearly  as  impor¬ 
tant  a  role  in  VTMR  estimation  as  nonstationarity,  as  we  will  show. 
The  effects  of  n<mstationarity  can  entirely  swamp  the  bias  by  in¬ 
ducing  estimation  error  in  the  opposite  direction,  i.e.,  overestimation. 

Figure  6.2  illustrates  tiie  effect  of  data  partitioning  on  the  standard 
deviation  of  the  VTMR  estimator  with  a  stationary  Poisson  process. 
Each  curve  represents  a  different  partitioning  of  the  ^ta  as 
explained  in  the  legmid.  The  coarsmr  the  partitioning,  the  hif^er  the 
standard  deviation  of  the  estimator.  Note  that  expected  total 
demands  has  very  little  effect.  The  sli^tly  smaller  values  associated 
with  expected  total  demands  of  five  are  probably  due  to  the  de&ult 
value  cf  1.0  being  assigned  to  the  estimator  for  the  random 
realizations  in  which  no  demand  occurred.  In  this  particular  &q>o- 
sition,  partitioning  alone  almost  triples  the  standard  deviation  of  the 
estimator.  The  current  system  operates  with  quarterly  data;  tWe- 
fiire,  it  is  vulnerable  to  the  phenomencm  shown  here.  A  finer  parti- 
timing  of  demand  data  could  decrease  the  standard  deviation  of  the 
VTMR  estimatmr. 


We  next  twramitm  a  nongtationary  prooeea  e^ioee  mean  changes  lin- 
eaxly  during  a  two-year  period  from  0.5  to  1.6  times  the  overall  mean. 
With  a  partitioning  of  the  data  into  24  equal  periods  (rou^y  oorre- 
apooding  to  mmiths)  and  eq>eeted  total  demao<to  of  SO,  for  example, 
the  mean  demand  rate  per  period  is  50/24  «  2.0633.  In  the  case  of 
this  ramp  process,  the  expected  demands  per  period  are  1.0417, 
1.1322, 1.2228, . . . ,  3.125.  Figure  6.3  shows  the  effects  of  partition¬ 
ing  and  expected  total  demands  <m  the  expected  value  ^  VTMR 
estimator  in  the  case  where  variation  around  the  changing  mean  fol¬ 
lows  a  Poissmi  process.  treatments  have  remarkable  effects. 

We  direct  the  readei^s  attention  to  the  middle  curve  in  Figure  6.3,  ie., 
the  curve  representing  a  partituming  (d'  the  data  into  ei|^t  periods,  as 
is  the  case  in  the  current  system.  The  length  of  the  periods  is  rouq^y 
a  fiscal  quarter.  Note  that  the  e]q>ected  value  of  the  VTMR  estimatm 
increases  finm  about  1  to  14  as  a  fiinctimi  of  the  total  demands  in  the 
two-year  period.  Althou^^  tlm  increase  in  the  mean  firom  0.5  to  1.5 
times  its  average  value  is  rather  steep,  it  does  serve  to  underscore  the 
dramatic  effect  that  nonstationarity  can  have  on  the  observed  VTMR. 

This  is  only  one  of  the  two  important  effects  reflected  in  this  illustra¬ 
tion.  Note  that,  for  an  item  with  1,000  total  demands  in  the  two-year 
period  (an  average  of  125  per  “quarter,”  an  unusually  hi^-demand 
item),  partitioning  of  the  data  has  a  dramatic  effect  on  the  expected 
value  of  the  VTMR  estimator.  Moving  firom  eif^t  periods  to  24  peri¬ 
ods  in  the  two  years  of  demands  reduces  the  eaq>ected  value  by  about 
two-thirds.  Even  as  we  move  toward  the  left  of  Figure  6.3  to  smaller 
values  of  total  demands,  finer  partitioning  of  the  data  still  has  mqjmr 
^ects  for  items  whose  demand  rates  are  m<M«  typical  of  those  in  the 
inventmy  system.  Of  course,  if  we  knew  that  the  underlying  mean 
was  a  ramp  process,  we  wouldn’t  estimate  the  VTklR  in  the  tradi¬ 
tional  way.  The  point  is  to  illustrate  that  the  direction  of  tiie  error  is 
typically  positive. 

The  combination  of  these  two  effects,  coarseness  of  partitioning  and 
nonstationarity,  can  result  in  some  very  large  values  of  the  VTMR  es- 
timatm*  when  the  demand  process  is  really  quite  well  behaved  around 
a  nonstationaiy  mean.  There  is  little  question  that  we  are  over-esti¬ 
mating  VTMRs  in  the  current  system  when  we  view  tiiose  VTMRs  as 
represoiting  the  stochastic  variability  in  the  demand  process.  The 
problem  is  that  in  estimating  spares  requirements,  stochastic  vari¬ 
ability  is  not  the  only  uncertainty  we  face  in  the  forecasting  proUem. 
We  also  face  all  of  ttm  other  sources  of  uncertainty  associated  with  es- 
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timatiiig  the  mean  dmnand  and  variability  ci  the  imMxsa  in  the  fu¬ 
ture,  i.e.,  at  the  end  of  the  procurement  lead  time. 

Our  principal  obaervationB  about  the  VTMR  estimator  are: 

•  The  variance  of  the  VTBfR  estimator,  rj,  increases  with  tiie  coarse¬ 
ness  of  the  partitioning  oi  tiie  observed  data  even  when  the  process 
is  stationaiy. 

•  Nonstationarity  acts  to  induce  overestimation  of  the  VHdR  as  a 
measure  <3i  variati<m  about  the  changing  mean. 

•  For  processes  of  the  type  discussed  here,  the  overestimation  in¬ 
creases  with  the  demand  rate  and  the  coarseness  of  the  partition¬ 
ing. 

•  Finer  partitioning  can  mitigate  the  effects  of  nonstationarity  on 
both  the  expected  value  and  variance  of  the  estimator. 

•  The  Idas  function  probably  has  little  or  no  effect  on  ^stem  perfor¬ 
mance. 

These  findings,  coupled  with  the  inference  firom  the  Bitbiug  data  dis¬ 
cussed  in  Section  3  that  there  is  some  level  of  nonstationarity  present 
in  these  dmnand  processes,  are  consistent  with: 

•  The  association  of  larger  values  of  observed  VTMRs  with  hi^- 
demand  items, 

•  The  substantial  differences  betwemi  the  distributions  of  VTMRs 
observed  in  the  Bitburg  data  and  those  discussed  by  Crawford,  and 

•  The  wildness  of  the  distribution  of  observed  VTMRs  around  the 
power  function  used  by  AFMC  to  specify  VTMRs  to  describe  the 
probability  distribution  of  the  numb^  of  items  of  each  type  in  re¬ 
supply. 

One’s  intuition  may  be  inappropriately  shaped  by  the  specific  exam¬ 
ple  of  the  ramp  process  discuss^  here.  Such  a  level  of  nonsta¬ 
tionarity  may  seldom  be  seen  in  real-world  parts  demand  processes; 
however,  it  clarifies  the  profound  importance  of  explicit  recognition  of 
tile  roles  of  nonstationarity,  data  pmrtitioning,  and  demand  rates  on 
the  distributions  of  observed  VTMRs.  It  seems  fairly  clear  that,  in 
general,  we  tend  to  overestimate  the  stochastic  variability  of  tiiese 
demand  proemses,  especially  those  of  hig^-demand  items.  In  the  dis¬ 
cussion  that  follows,  we  point  out  the  need  to  consider  explicitly  all 
the  other  sources  of  uncertainty  that  affect  the  demand  process  in  our 
estimation  of  variance. 
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mPBOVED  VABIANCE  bshmahon 

The  variance  of  the  d^and  process  is  one  of  the  two  most  important 
elements  of  the  forecasting  problem.  Our  estimaticm  of  the  variance 
may  wdl  have  greats  effects  on  the  performance  of  the  syrtem  than 
our  estimatkm  of  the  mean.  Variance  estimatum  tends  to  be  a  bit  ne¬ 
glected  relative  to  its  impmtance,  owing  to  the  traditional  logic  of  im- 
derstanding  the  mean  process  before  the  variance  of  demands  around 
the  mean  is  addressed.  The  discussion  of  varial^ty  in  Section  4  fo¬ 
cuses  attention  on  several  separate  components  of  the  variance  of 
interest  in  this  discussion.  The  fundamental  logical  proUem  underly¬ 
ing  variance  estimation  is  to  clarify  our  assumptions  about  mmsta- 
tionarity  and  to  explicate  the  differmice  between  stochastic  variability 
and  forecast  variance. 

If  we  assume  stationarity,  then  the  current  variance  estimation  pro¬ 
cedure  implicitly  tries  to  estimate  stochastic  variability.  It  then  uses 
this  estimate  as  an  estimate  of  forecast  variance.  If  these  demand 
processes  really  were  stationary,  then  the  current  procedure  would 
actually  underestimate  the  forecast  variance  somewhat  because  the 
forecast  variance  should  include  both  parameter  variance  and 
stochastic  variability.  The  problem  is,  if  there  is  nonstationarity  pre¬ 
sent  in  these  processes,  then  the  current  procedure  overestimates 
stochastic  variatnlity  because  it  fails  to  account  for  the  nonstationar¬ 
ity.  To  make  matters  worse,  the  current  procedure  then  uses  this 
poor  estimate  of  stochastic  variabilily  as  an  estimate  of  forecast  vari¬ 
ance.  But  with  nonstationarity  present,  under  an  explicit  assumption 
of  nonstationarity,  the  forecast  variance  should  include  not  only 
stochastic  variability  and  parameter  variance  but  should  also  account 
for  the  effects  of  the  nonstationarity.  Because  it  overestimates 
stochastic  variability  and  fails  to  indude  other  sources  of  variability* 
the  current  procedure  enjoys  the  effects  of  errors  in  the  opposite  di¬ 
rection  that  don’t  necessarily  cancel  out;  i.e.,  it  isn’t  as  bad  an  esti¬ 
mate  of  forecast  variance  as  it  mig^t  be.  Interesting^,  this  is  so  quite 
by  chance. 

The  apiMTopriate  variance  to  calculate  is  a  function  of  the  dedsion  be¬ 
ing  made.  Here  we  focus  on  the  forecast  variance  assodated  vnth 
three-year  procurement  lead  times.  Decisions  whose  effects  are  felt 
over  Sorter  planning  horizons  (e.g.,  annual  repair  planning)  would 
be  less  vulnerable  to  forecast  uncertainty  ovdng  to  the  reduced  effects 
of  nonstationarity  across  the  shorter  planning  horizon.  Once  we  have 
eiqilicated  precisely  what  forecast  variance  we  are  calculating,  we  can 
consider  how  to  estimate  the  variance  and  how  well  it  can  be  det«^ 
mined.  Unfortunately,  on  the  latter  count  we  are  in  no  better  a  posi- 
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tion  with  forecast  variances  than  we  were  with  process  variances; 
these  are  highly  variaUe  processes  and  their  variances  are  difficult  to 
estunate. 

We  should  also  be  dear  about  predsely  what  we  mean  1^  nonstation- 
arity.  In  additi<Hi  to  nonstationarity  in  the  mean,  we  could  also  have 
nonstationarity  in  the  stochastic  variability.  As  a  matter  of  fact, 
there  is  every  reason  to  believe  that  thia  is  true.  There  is  also  every 
reason  to  believe  that  we  will  be  unable  to  do  a  good  job  of  estimating 
this  nonstationarity.  As  we  have  shown  in  earlier  sections,  the  vari¬ 
ance  of  the  VTMR  estimator  is  large  even  for  stationary  processes. 
We  have  an  even  more  difficult  problem  in  the  face  of  nonstationarity 
in  the  stochastic  variability.  The  methods  we  propose  here  do  not 
explidtly  incorporate  this  nonstationarity. 

As  in  any  prddem  where  we  have  inadequate  information  to  estimate 
variances  confidently,  we  must  look  for  additional  information  to  en¬ 
hance  the  estamatiim  process.  The  information  mi|^t  come  firom 
other  parts  or  other  relationships.  The  current  system’s  method  of  es¬ 
timating  variances  is  to  look  at  the  relationship  between  means  and 
VTMRs  across  many  parts  and  then  use  the  (presumably)  better  es¬ 
timated  means  to  spe^  the  variances.  There  are  other  ways  to  im¬ 
prove  our  estimates  by  pooling  information.  Parts  could  be  grouped 
together  in  various  ways  and  their  variances  jointly  estimated,  as 
with  ’’shrinkage”  estimators,  for  example.  More  data  could  be  ob¬ 
tained  through  more  detailed  data  collection  either  by  partitioning 
the  data  more  finely  in  time  or  by  mnng  base-specific  data.  Finally, 
other,  better  estimated  quantities  could  be  used  to  develop  more 
finely  tuned  estimators  of  variance  along  the  lines  of  the  present  eys- 
tmn’s  VTMR  estimator. 

We  were  unable  to  explore  some  of  these  methods  because  we  did  not 
have  access  to  data  partitioned  more  finely  than  by  quarter,  or  to 
base-specific  data.  The  effects  of  finer  partitioning  of  the  data  are 
suggested  ly  results  laesented  above.  We  think  this  is  a  potentially 
firuitful  area  for  improvement.  Base-specific  data  woiild  not  only  be 
useful  for  variance  estimaticm  but  for  stock  level  and  asset  allocation 
decisionmaking  as  well. 

It  is  important  to  mention  ideas  we  explored  that  did  not  pan  out. 
One  seductive  possibility  is  to  produce  a  model  like  the  current  sys¬ 
tem’s  nonlinear  function  for  VTMR  estimation  that  incorporates  addi¬ 
tional  explanatoiy  variaUes.  In  particular,  there  seems  to  be  a  It^cal 
disconnect  in  the  current  system.  The  current  system  would  take  two 
parts  with  radically  different  historical  variability  and  i»redict  the 
same  variance  if  tiiey  had  the  same  mean.  An  obvious  fix  for  this  dis- 


connect  ia  to  devdop  a  model  for  the  variance  that  incorporates  his¬ 
torical  variability  as  well  as  historical  means.  The  problmn  hmre  is 
what  to  use  as  a  measure  a[  historical  variability.  We  already  know 
that  the  variance  is  poorly  estimated  1^  the  VTMR  estimator. 
Indeed,  we  can  find  a  statistically  significant  relationship  between 
historical  variability  and  future  variability  even  after  controlling  for 
the  mean.  But  an  estimator  based  on  this  relationship  does  not  worir 
well.  The  varial^ty  in  the  variance  estimates  is  just  too  large. 

With  the  lack  of  more  finely  partitioned  data  and  our  inability  to  in¬ 
corporate  part-specific  variance  estimates  into  the  forecast  variance, 
we  are  left  with  only  one  option.  We  must  explore  a  mean-variance 
relationship  such  as  the  one  used  in  the  current  system  and  see  if  we 
can  improve  it.  Thmre  is  reason  to  believe  that  we  can  improve  on  \e 
current  system’s  nonlinear  function.  Our  hopes  rest  on  the  conse¬ 
quences  of  our  explicit  assumption  of  nonstationarity.  If  we  consider 
nonstationarity,  the  current  system  is  estimating  the  wrong  variancs. 
Hence  if  we  apply  the  current  system’s  general  approach  of  using  a 
nonlinear  regression  function  to  the  three-year  forecasts,  we  miglit 
improve  the  VTMR  estimator. 

To  develop  the  improved  variance-to-mean  relationship  we  can  take 
the  three-year  forecast  errors  and  try  to  characterize  their  magnitude 
as  a  fimction  of  their  forecast  means.  The  method  here  is  to  make 
forecasts  with  whatever  method  one  intends  to  use,  calculate  the 
three-year  forecast  errors,  then  fit  a  nonlinear  function  in  a  manner 
similar  to  that  of  the  current  ^stem  to  forecast  the  variance  (or 
VTMR). 

Our  best  mean  forecaster  is  the  wei^ted  regression  forecaster  for 
Group  1  and  the  current  system’s  ei^t-quarter  moving  average  for 
Groups  2  and  3.  We  appUed  these  methods  to  our  sample  of  600 
parts.  We  made  forecasts  for  10, 11, 12,  and  13  quarters  in  the  future 
and  for  each  of  those  quarters  computed  the  observed  squared  error  of 
the  forecast.  This  corresponds  rou^y  to  a  three-year  procurement 
lead  time.  We  then  performed  a  simple  linear  regression  of  the  log  of 
the  squared  forecast  errors  on  the  log  of  the  forecast.  This  gives  us  a 
variance-to-mean  relationship  for  our  particular  three-year  forecast¬ 
ing  method.  This  could  be  readily  applied  to  other  forecasts  needed 
for  plaiming  and  procurement.  Indeed,  this  relationship  should  be 
reestimated  each  year.  This  is  especially  important  in  times  of  signif¬ 
icant  changes  in  funding  levels,  system  structure,  activity  levels,  or 
other  factors  that  could  affect  future  pipelines.  It  could  also  be  tai¬ 
lored  to  item-specific  procurement  lead  times.  Note  that  the  variance 
now  includes  some  of  the  uncertainty  stemming  from  nonstationarity. 
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The  magnitude  of  this  variance  could  well  be  affected  by  system  re¬ 
structuring. 

Our  repression  yielded  the  relationdiip: 

VTMR  =  0.57  MEAN®  ", 
constrained  to  be  between  1.01  and  5.0. 

Other  ezfdanatory  variables  mi^t  be  useful  in  estimating  VTMRs, 
for  example,  the  QPA  (as  we  observed  in  the  Bitburg  data);  however, 
the  QPA  is  specific  to  an  item’s  application  raffier  than  to  the  item  it- 
sdf  and  is  of  limited  utility  at  the  aggr^^ate,  systemwide  level  where 
these  estimates  are  being  made. 

The  ultimate  measure  of  the  effica<y  of  improved  demand  and  vari¬ 
ance  forecasting  is  the  magnitude  of  its  improvement  in  system  per- 
finmanoe.  In  the  section  that  follows,  we  discuss  our  evaluations  of 
these  improved  techniques. 


7.  EVALUATIONS  OF  THE  IMPROVED  DEMAND 
FORECASTING  AND  VARIANCE 
SPECIFICATION  TECHNIQUES 


As  we  pointed  out  above,  600  items  were  used  to  develop  the  im¬ 
proved  demand  forecasting  and  variance  specification  techniques, 
only  200  of  which  were  hig^-demand  (15  or  more  demands  per  qiiar- 
ter)  items.  We  then  tested  these  methods  on  aU  hi|^-demand  parts  in 
the  dataset,  4,215  items.  The  improvements  in  root  mean  squared  er¬ 
ror  and  mean  absolute  deviation  achieved  by  using  the  improved 
techniques  in  contrast  to  the  current  system  are  reflected  in  Table 
7.1.  The  improvement  is  impressive.  (Also  see  Figures  5.3  and  5.4.) 

Beyond  evaluating  the  improved  demand  and  variance  finecasting 
techniques  using  ordinary  statistical  measures,  it  is  important  to  un¬ 
derstand  how  much  they  might  improve  the  cost-effectiveness  of  the 
spares  procurement  miv  over  a  realistically  long  planning  horizon. 
We  developed  a  system  of  software  that  replicates  AFMC’s  spares  re¬ 
quirements  computation  and  central  stock  leveling  i^stem  and  evalu¬ 
ates  the  performance  of  the  resulting  spares  stockage  posture.  We 
used  this  software  system  to  evaluate  the  imp;  wed  demand  forecast¬ 
ing  and  variance  specification  techniques.  The  system  is  described  in 
Figure  7.1  (except  that  the  WRSK  requirements  portion  of  the  sofl^ 
ware  was  not  used  in  these  evaluations).  To  evaluate  these  tech¬ 
niques,  we  used  the  March  1986  requirements  database  used  by 
AFMC  to  compute  spares  and  repair  requirements  and  replicated  the 
requirements  computation,  first  with  the  current  system’s  forecasting 
techniques  and  then  with  the  improved  tediniques.  We  input  the  re¬ 
sults  to  a  replica  of  AFMC’s  central  stock  leveling  qrst^  (D028), 
which  allocates  stock  levels  to  bases  and  the  depot,  and  added  the  war 
readiness  spares  the  units  are  authorized,  thereby  estimating  the 

Table  7.1 

Percentage  of  Inqirovement  in  RMSE  and 
MAD  of  Improved  Techniques  over  Current 

System 


Measure  10-Quarter  Horizon  13-Quarter  Horizon 

RMSE  48  38 


Figure  7.1 — Schemetic  Repreeentation  of 
EvahuitioB  Syetem  Software 


anticipated  stockage  posture  that  would  eventuate  in  the  system 
rou^dy  three  years  (actuaUy  13  quarters)  after  the  requirements 
oompiftation.  We  then  evaluated  four  stockage  postures:  (1)  One 
anticipated  to  result  from  the  current  aystem  using  the  same  aircraft 
availalnlity  goals  as  used  in  the  current  qrstem,  (2)  one  anticipated  to 
result  fium  use  of  the  improved  techniques  using  the  same  specified 
goals,  (3)  <me  anticipated  to  result  firom  the  first  approach  coupled 
with  roug^tdy  a  ^40  million  budget  reduction,  and  (4)  one  anticipated 
to  result  from  the  improved  tedimques  coupled  with  a  budget  rou^y 
equal  to  that  in  case  3.  In  these  evaluations,  war  readiness  spares 
were  not  induded.  Using  the  case  of  the  F-16  aircraft,  we  evaluated 
each  of  these  stockage  postures  with  an  advanced  capability  assess¬ 
ment  model,  Dyna-METRIC  Version  6,  under  several  sets  of  assump¬ 
tions  diat  we  will  describe. 

The  evaluations  of  the  four  stockage  postures  were  done  with  item 
characteristics  drawn  from  the  March  1990  database.  The  database 
contains  eight  quarters  of  past  history  of  demands,  NRTS  actions, 
and  so  f<nth.  Those  data  were  used  to  evaluate  the  stodcage  postures 
antidpated  to  result  from  the  two  alternative  forecasting  methods  in 
use  with  the  March  1986  database.  Thus  the  world  eventuated  dif¬ 
ferently  than  antidpated  at  the  time  the  requirements  computation 
was  done,  and  the  differences  are  ezplidtly  accounted  for  in  these 
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evahiationB.  The  demand  rates  and  variances  actually  observed  finmi 
A|iril  1988  throuf^  March  1990  wtn  used  f(nr  the  evaluati<«is  except 
that  variances  were  oonstrained  to  be  less  than  or  equal  to  15.  These 
overliq)  the  point  in  tune,  June  1989,  that  was  an  average  lead 
time  beyond  the  buy  point  for  the  FY87  budget.  The  results  of  the 
evaluati<ms,  shown  in  Tables  7.2  and  7.3,  are  disappointing  in  the 
sense  that  the  effiacts  the  improved  techniques  are  largely  masked 
hy  the  much  more  dramatic  effects  of  the  managmnent  adaptations 
assumed  to  be  in  place  in  three  of  the  scenarios. 

In  Tables  7.2  and  7.3,  the  case  labeled  “No  cannibalization*  repre¬ 
sents  the  situati<m  in  which  there  is  no  omsolidation  of  parts  short¬ 
ages  (cannibalization)  among  aircraft.  The  second  case,  labeled  *Full 
cannibalization”  reflects  the  assumption  in  the  evaliiations  of  canni¬ 
balization  of  aU  parts  shivtages  that  increase  aircraft  availability. 
The  third  case  adds  another  management  adaptation,  that  of  laterid 
supply.  The  fourth  case  adds  a  more  responsive  depot  repair  system, 
one  that  expedites  the  transportation,  handling,  and  processing  of 
components,  reducing  pipeline  times  firom  an  average  of  89  days  to  an 
average  of  56  days  for  overseas  bases  and  48  da3rs  for  CONUS  bases. 

Three  points  are  worthy  of  note  in  Tables  7.2  and  7.3.  The  first  is 
that  the  investment  level  and  system  performance  that  result  finm 
spedtymg  the  aircraft  availability  goals  in  the  traditional  way  (Table 

7.2)  are  superior  with  the  improved  methods  because  they  deliver 
somewhat  better  performance  with  $76  million  less  budgetary  re¬ 
quirement.  The  second  point  is  that  for  rou^y  equal  budgets  (Table 

7.3) ,  the  improved  teefoniques  deUvmr  generally  better  performance. 
The  final  point  is  that  the  performance  with  tiie  improved  methods 
and  reduced  budget  is  almost  as  good  as  that  of  the  current  ^stem 
with  an  unreduced  budget;  however,  the  evaluations  are  somewhat 
difficult  to  interpret  because  all  four  of  these  cases  are  on  such  a  flat 
part  the  availability/cost  curve;  ie.,  substantial  budgetary  changes 
produce  relatively  litiJe  effect  on  performance,  suggesting  that  the 
stockage  and  budgetary  requirements  being  computed  by  the  current 
system  are  quite  rich. 

Given  a  different  database  (e.g.,  March  1987  or  1988),  the  outcome  of 
this  experiment  mig^t  have  been  somewhat  different,  of  course. 
What  we  have  shown  is  that,  /or  any  given  budget  level,  the  improved 
technique  are  likely  to  deliver  improved  performance  simply  because 
they  substantially  reduce  the  expected  forecasting  error.  Forecasting 
errrar  acts  exactly  like  chxum  does  in  its  ^fects  on  system  perfor¬ 
mance.  It  is  another  realization  of  a  future  that  we  did  not  expect 
when  we  specified  item  parameters  to  the  requirements  computation. 
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TiOileTJ 

Co«t  and  Pertoraumce  wHh  Tr«dHtioiud  Availability  GoaLi 


Manacement 

Admitatioiia 

POroantage  (rf  Aircraft 
Unavailablo,  Poacetmie 

Current  Systam, 
$3,709  Millkm 

Inquoved  Mediods, 
$3,633  Million 

No  cannihaliaatkin 

74.9 

71.7 

33.0 

32.3 

Cannibaliaatkin,  lataral  supply 

17.3 

16.4 

CaaoibaUxatioa,  qui^  latei^  supply 

3.2 

3.1 

Table  7^ 

Cdat  and  Petfonuaioe  with  Bednced  Bmlgeta 


Managsment 

Peroantaga  of  Aircraft 
Unavailable,  Peacetime 

Current  System, 
$3,474  Million 

Improved  Methods, 
$3,470  Million 

No  cannibalisatkin 

81.6 

76.3 

Foil  cannibaKsatioQ 

34.6 

33.1 

Cannibalisation,  lateral  supply 

19.0 

17.2 

Cannibaliaation,  quid^  lataral  siqiply 

3.5 

3.6 

In  these  evaluations,  the  improvements  in  variance  specification  and 
forecasting  translate  to  improved  system  performance  as  well. 

Note  that  we  did  not  explicitly  evaluate  the  improved  variance  speci¬ 
fication  technique  independently.  One  would  expect  very  little  effect 
firom  moving  to  the  use  of  the  new  variance  specification  simply  be¬ 
cause  it  is  so  much  Kka  the  formula  currently  in  use.  The  point 
should  be  made  again,  however,  that  what  is  important  about  the  new 
technique  is  that  it  explicitly  considers  forecasting  imcertainiy  and  is 
smisitive  to  the  length  of  the  planning  hmizon.  Thus  its  underlying 
logic  is  quite  different  fix»m  the  logic  of  the  current  variance  specifica¬ 
tion  method.  It  is  this  difference  that  is  important  rather  than  the 
specific  numerical  values  in  the  formulas. 

It  is  also  important  to  mite  that  the  budgetary  values  shown  in  these 
evaluations  apply  only  to  Uie  first  3rear  of  implementation.  It  is  not 
dear  what  the  savings  in  subsequent  years  would  be  relative  to  those 
in  the  first  year. 


8.  CONCLUDING  REMARKS 


We  coi^ade  from  this  research  that: 

•  A  wei(^ted  regression  forecasting  technique  would  perform  better 
than  the  ei^t-quartm'  movmg  average  for  high-<temand  items, 
that  is,  for  items  with  IS  or  more  demands  per  quarter. 

•  A  modifieati<m  to  the  power  foiwticm  currently  used  AFMC  to 
specify  the  variance  of  the  number  of  items  in  resupply  would  yield 
an  improved  mix  of  spares  for  a  specified  investment  level 

•  A  NUTS  forecaster  that  incorporates  exponential  wei^ting  of  past 
NUTS  rates,  coupled  with  the  improved  demand  forecaster,  would 
improve  forecasts  of  NBTS  actions  that  are  important  to  depot  re¬ 
pair  planning. 

These  three  improvements  could  3deld  substantial  savings  in  spares 
costs  for  specified  levels  ^stem  performance.  In  the  spares  re¬ 
quirements  computation  done  with  ^e  March  1986  database,  these 
changes,  coupled  with  an  investment  level  $239  million  less  than  the 
current  system’s,  yielded  rou^y  the  same  system  performance. 

We  recommend  implementation  oi  the  wei^ted  regression  forecaster 
with  an  a  of  0.75  and  a  VTMR  specification  of  0.57  con¬ 

strained  to  be  in  the  range  of  1.01  to  5.0.  The  0.75  wei^ting  factor  is 
also  recommended  for  the  improved  NSTS  rate  forecaster. 

In  the  D035C  data  system,  supply  transactions  are  reported  daily; 
therefore,  there  may  1^  some  way  to  move  to  the  use  of  a  finer  parti¬ 
tioning  of  demand  data  than  is  used  in  the  current  system.  This 
would  make  the  VTMR  estimator  considerably  less  vulnerable  to  non- 
stationarify.  For  some  purposes,  quarterly  historical  demand  data 
may  be  sufficient,  but  for  purposes  of  variance  estimation,  more  finely 
partitimied  data  would  clearly  be  more  helpful. 

In  retrospect,  we  view  the  recommendations  above  as  only  one  impor¬ 
tant  outcome  of  this  research.  The  explicit  recognition  and  treatment 
of  nonstationaiify  in  these  parts  demand  processes  are,  perhaps, 
more  important  ideas  in  the  longer-term  schmne  of  things.  In  future 
research  in  this  area,  this  ctmsideration  is  especially  important.  Any 
future  imjnovements  in  spares  demand  modeling  and  finrecasting  will 
probably  depend  on  these  ideas. 
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This  research  leavm  unresolved  one  of  the  most  important  problems 
complicating  spares  and  repair  requirements  estimation:  the  estima- 
tion  ci  wartime  demand  rates.  We  do  not  intend  to  imply  by  this  that 
we  think  we  have  resolved  all  other  problmns,  but  we  do  believe  that 
the  results  reported  here  represent  a  significant  improvement  to  the 
tq>proach  to  demand  forecasting  taken  by  the  current  system. 

We  recommend  the  following  topics  for  future  research: 

•  Extensions  of  the  analysis  of  base-level  demand  processes  to  addi¬ 
tional  bases  and  weapmi  qrstems, 

•  Explorati<m  of  the  use  of  empirical  Bayes  estimators  in  the  AFMC 
Cmitral  Leveling  System  (D028),  and 

•  Examination  of  alternative  methods  for  modeling  the  demands  for 
low-demand  items. 

As  pointed  out  in  Section  3,  our  observations  about  base-level  demand 
processes  ware  based  on  the  Bitbuig  data.  The  analyses  of  these  data 
need  to  be  extended  to  other  bases  and  weapon  systems.  The  motiva¬ 
tion  for  these  extensions  is  to  ensure  that  we  are  modeling  base-level 
demand  processes  sensibly  in  decision  support  ^sterns  used  centrally. 

It  has  long  been  hypothesized  by  Crawford  and  others,  including  the 
currant  autlmrs,  that  neither  base-specific  nor  worldwide  demand 
rates  are  the  best  estimators  of  base-specific  future  demands. 
Preliminary  evaluations  of  empirical  Bayes  estimators  that  adjust 
base-specific  demand  rates  by  worldwide  means  tend  to  reinforce  this 
hypothesis;  however,  the  sample  size  of  these  evaluations  was  too 
small  to  be  ocmdusive,  although  they  served  to  strengthen  our  intu¬ 
ition  that  the  hypothesis  is  a  sensible  one.  Unfortunately,  the  abiliiy 
to  evaluate  this  proposed  approach  would  depend  on  a  data-collection 
effort  that  could  be  quite  extensive  and  time  consuming.  The  problem 
is  that  data  describing  base-specific  demands  over  time  are  needed  to 
support  the  evaluations.  This  requirement  could  conceivably  be  sup- 
pmted  by  NRTS  actions  reported  in  the  Maintenance  Data  Collection 
System,  althoui^  base  supply  transaction  data  woxild  decurly  be  bet¬ 
ter  for  die  purpose.  We  have  been  led  to  believe  that  such  data  could 
be  collected  throu|^  die  D024  system,  but  the  effort  would  require  at 
least  four  years  of  transaction  data  fium  several  bases.  Althou^^ 
sudi  a  data-ooUecdon  effort  seems  quite  amtntious,  the  data  could 
serve  seversd  interests,  induding  central  stock  leveling  techniques. 

One  of  the  prddems  with  the  approach  to  the  weifi^ted  regression 
tedinique  discussed  here  is  diat  it  sissumes  normal  distributions  of 
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demands.  While  this  may  be  a  reasonable  approximation  for  hi^- 
demand  items,  it  is  flawed  as  a  model  for  lower-demand  items,  espe¬ 
cially  for  itmns  with  very  low  demands.  We  regret  not  being  able  to 
suggest  additi<mal  avenues  of  exploration  that  hold  promise  of  success 
with  low-demand  items,  but  hypotheses  should  be  encouraged  and 
evaluated  even  thou^  low-demand  items  typically  have  less  impact 
on  system  performance. 

Another  important  problem  involved  with  modeling  demands  for  low- 
demand  items  is  that  past  demand  data  provide  too  little  information 
about  the  demand  processes  of  individual  items.  Thus  a  second  idea 
emerges:  to  borrow  strength  finm  the  demand  history  of  many  low- 
demand  items  to  enhance  our  abilily  to  model  demands  for  individual 
items.  This  idea  seems  as  reasonable  now  as  when  it  first  emerged 
from  earlier  RAND  research.  Unfortunately,  it  was  never  pursued 
with  sufddent  thorou^mess  to  lead  to  implementation.  The  idea  of 
pooling  data  across  items  has  Bayesian  roots,  of  course,  as  does  the 
Kalman  filter.  They  are  compatible  in  philosophy  and  could  be  com¬ 
bined  with  empirical  Bayes  approaches  to  central  leveling.  This 
would  constitute  a  unified  Bayesian  approach  to  demand  modeling  to 
support  the  central  allocation  of  stock  levels  as  well  as  spares  and  re¬ 
pair  requirements  estimation. 


i^pendix  A 

RELATIONSHIP  OF  THE  KALMAN  FILTER  TO 
OTHER  FORECASTING  METHODS 


One  useful  characteristic  of  Kalman  filter  models  is  the  way  they 
generalize  other  well  known  time  series  and  forecasting  methods. 
Unfortunately,  this  has  caused  some  confusion  in  the  literature.  It  is 
not  correct  to  claim  that  Kalman  filtering  is  ‘^ust  exponential  smooth¬ 
ing,”  as  is  sometimes  heard.  Exponential  smoothing  is  a  special  case 
of  Kalman  filtering  but  Kalman  filtering  incorporates  a  much  richer 
collection  of  models.  In  the  discussion  that  follows,  we  demonstrate 
how  some  well-known  statistical  models  can  be  written  as  Kalman  fil¬ 
ter  models.  Refer  to  Ref.  63  for  a  more  complete  exposition. 

1.  Aflrst  order  autor^pressive  process  in  the  state  equation. 

The  state  equation  is 

Pt . 

The  measurement  equation  is 


Some  useful  facts: 

1.  corr  (Pt.Pt-j)  =  . 

2.  If  <|)  =  0  this  is  a  random  effects  model, 

3.  If  <|>  =  1  this  is  the  “simple”  Kalman  filter  model. 

n.  A  first  order  vector  autoregressive  process  in  the  state 
equation. 

Replace  p  and  with  vectors  and  ^  with  a  square  matrix  in  example  I. 

m.  A  vector  autoregressive  moving  average  process  in  the 
state  equation,  p  and  w  are  vectors  and  ^  and  6  are  square 
matrices. 
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The  state  equation  is 

Pt  ~  P  ~  ♦liPt-i  ~  P) - ^p(Pt-p  ~  P) 

=  Wt-0iWt_, - e,wt_,  , 

where  p  is  the  number  of  observations  underlsring  the  moving  average 
and  q  is  the  number  of  steps  in  the  autoregression. 

IV.  Example  1  is  equivalent  to  certain  linear  models. 

The  state  equation  is 


Pt-P  =  «(Pt-i-P)  +  Wt  . 

The  measurement  equation  is 


Yt=XP,  +  Vt  . 

Now  manipulate  the  measurement  equation 

Yt  ={Xp  +  (Xp-Xp)  +  Vt  . 

Let  (XPt  -  Xp)  +  Vj  5=  af  So  Y^  *  Xp  +  at,  where  at  is  hetero- 
skedastic  and  autocorrelated.  If  the  struct^  and  values  of  the 
elements  of  at’s  variance-covariance  matrix  are  known,  the  param¬ 
eters  can  be  found  using  generalized  least  squares. 

V.  The  simple  Kalman  filter  model  is  similar  to  an  exponential 
smoother. 

The  state  equation  is 


®t  =  0t-i  +  • 

The  measurement  equation  is 


Yt  =  Ot  +  Vt  . 


This  model  corresponds  to  a  class  of  ARIMA(0,1,1)  models  firom  Box 
and  Jenkins  [64]. 
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If  we  set  Vt  -  2  and  =  1  we  can  use  the  update  equation 

to  solve  for  6^  as  a  function  of  the  observations  and  initial  conditions 

This  reveals  the  filter  to  be  a  simple  exponential  smoother  with  a 
starting  value  incorporated.  Of  course,  the  Kalman  filter  provides 
standard  errors,  an  intuitive  reason  for  the  smoothing  parameter,  and 
obvious  wa3rs  to  extend  the  model  if  the  fit  is  inadequate. 


.^ypendixB 

THE  FUNCTIONAL  FORM  OF  THE  MEAN 
DEMAND  PROCESS 


Although  the  simple  multiplicative  function  for  the  mean  demand 
rate  that  the  current  S3r8tem  uses  is  incorrect,  finding  the  best  alter¬ 
native  is  a  difficult  problem.  The  process  of  selecting  a  model  based 
on  the  data  is  itself  not  a  well  understood  problem  even  for  station¬ 
ary,  normal  models.  The  approach  we  take  here  is  to  conduct  an  au¬ 
tomated  model  search  on  the  sample  of  parts  used  in  Section  4. 

A  DATA-ANALTnC  APPROACH  TO  DETERMINING  THE 
FUNCTIONAL  FORM 

We  first  examine  a  linear  regression  model  for  the  mean  demand  as  a 
function  of  flying  hours,  flying  hours  in  the  previous  period,  and  de¬ 
mands  in  the  previous  period.  Intuitively  these  quantities  reflect 
three  sources  of  demand.  Flying  hours  reflect  current  stress  on  the 
aircraft.  Flying  hours  firom  the  previous  period  reflect  the  delayed  ef¬ 
fect  of  earlier  stresses.  Indudi^  demands  fiium  the  previous  period 
is  an  effort  to  capture  stresses  unrelated  to  flying  hours  with  a  dura¬ 
tion  longer  than  one  quarter.  The  idea  of  using  lagged  demand  is  that 
there  may  be  real,  unmodeled  stresses  on  the  system  reflected  by  pre¬ 
vious  demand.  A  fourth  source  of  demand,  a  teckground  level  of  de¬ 
mand  imrelated  to  use,  is  reflected  by  the  inclusion  of  a  constant  in 
the  model.  In  this  section  we  refer  to  these  explanatory  variables  as 
X,  to  have  a  generic  label. 

We  consider  three  possible  ways  to  modify  this  basic  model:  (1)  se¬ 
lecting  a  subset  of  these  explanatory  variables,  (2)  transforming  the 
explanatory  variables,  and  (3)  transforming  ffie  demands.  Unfor¬ 
tunately,  the  order  in  which  we  consider  these  possible  modifications 
may  have  an  effect  on  our  conclusions.  For  example,  if  we  decide  to 
transform  the  demands  by  taking  the  logarithm,  we  may  select  differ¬ 
ent  explanatory  variables  than  if  we  had  not  transformed.  We  will 
use  a  model  search  strat^y  that  should  yield  a  model  with  good  pre¬ 
dictive  qualify  [65].  That  strategy  has  four  steps:  (1)  select  the 
explanatory  variables,  (2)  examine  the  data  for  potential  outliers,  (3) 
transform  the  demands  if  needed,  and  (4)  transform  the  explanatoiy 
variables  if  needed. 
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In  the  discussion  that  follows,  we  explain  the  automated  procediues 
used  for  each  of  these  stages,  present  the  results  of  this  strat^y  for 
model  selection,  and  consider  the  plausibility  of  these  results  and 
other,  related  models. 

DESCRIPTIONS  OF  THE  COMPONENT  PROCEDURES 
Mallow’s  Cp 

Mallow’s  Cp  is  a  criterion-based  variable  selection  method  [66].  The 
idea  is  to  minimize  the  mean  squared  error  of  prediction,  Jp,  where  Jp 
is  given  by 


where  n  is  the  number  of  observations. 

Of  coiirse,  there  are  some  unknown  quantities  here.  As  an  estimate 
of  Jp,  Mallows  [66]  proposed 

^  RSSp  „ 

Cp  =  +  2p  -  n  . 

RSSp  is  the  residual  sum  of  squares  from  a  model  with  p  explanatory 
variables,  and  is  the  error  estimate  from  the  full  model.  The 
*best”  model  is  the  one  with  the  smallest  Cp.  This  method  of  variable 
selection  is  intended  to  produce  models  that  predict  well.  The  details 
of  the  computation  can  be  found  in  Ref.  67,  pp.  215-217. 

Outlier  Rejection  Using  the  Bonferroni  Inequality 

This  outlier  test  is  quite  intuitive.  The  regression  is  refit  with  the 
questionable  point  omitted.  Under  the  usual  regression  assumptions 
the  difference  between  the  omitted  point  and  the  refitted  line  can  be 
compared  using  the  standard  error  for  prediction  based  on  the  refitted 
equation.  The  actual  calculation  is  as  follows: 

First,  the  residuals  are  calculated  for  eadi  case  by  fitting  the  model 
with  the  case  deleted,  and  the  t  ratio  is  formed  to  test  the  hypothesis 
that  the  residual  is  zero.  This  calculation  can  be  done  by  using  the  re¬ 
lationship 
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ti= - P—  . 

0(j)-y/l-  hj 

In  this  expression,  Sj  represents  the  raw  residual  for  the  ith  case  and 
hi  is  the  diagonal  of  the  *hat”  matrix,  X  (X'X)~'  X'.  Theti  are  the 
externally  studentized  residuals,  althou^  this  terminology  is  not 
universal.  0/|^  is  estimated  from  the  regression  with  case  i  omitted. 
All  (rf'  these  tj  are  t-distributed  with  n  -  p  -  1  degrees  of  freedom. 
The  tj  with  the  largest  absolute  value  is  then  tested  to  see  if  it  is  too 
large.  Since  we  are  picking  the  largest  of  n  residuals,  we  need  to  get 
a  critical  value  for  the  largest  absolute  value  of  n  draws  from  the 
student’s  t  distribution.  Snedecor  and  Cochran  [68]  suggest  testing 
the  largest  residual  at  the  level  o/n  against  the  t  distribution  with  n  - 
p  -  1  degrees  of  freedom.  This  Bonferroni  procedure  guarantees  a 
test  of  no  more  than  size  oe. 

It  is  not  actually  necessary  to  recompute  the  regression  with  each  ob¬ 
servation  omitted.  There  are  more  convenient  formulas.  All  of  this 
material  is  covered  in  Ref.  67,  pp.  114-117. 

Box-Cox  Procedure 

The  Box-Cox  procedure  [69]  selects  a  transformation  of  D  automati¬ 
cally  friom  a  collection  of  power  transformations.  The  Box-Cox  model 
assumes  there  is  a  power  transformation,  indexed  by  the  power  X., 
such  that  the  usual  linear  model  assumptions  are  satisfied,  i.e., 

D*’  =  XP  +  e, where e is N( 0,0®) . 

In  this  implementation,  the  likelihood  frmction  is  calculated  for  X.  in 
the  set  (-2,  -1,  -0.6, 0, 0.5, 1, 2],  where  the  power  zero  transformation 
corresponds  to  taking  the  logarithm.  The  value  that  maximizes  the 
likelihood  on  this  set  is  the  selected  transfrirmation.  Note  that  no  test 
is  done  to  see  if  a  transformation  is  required. 

This  procedure  requires  D  to  be  positive.  If  any  value  is  zero,  all  D 
values  are  shifted  so  that  the  smallest  value  is  1.0. 

B<KK-Tidwdn  Procedure 

The  Bmc-Tidwell  procedure  [62]  provides  an  autcnnated  method  of 
transfr>nning  the  Xs.  Consider  the  model  for  the  transformation  oi 
Xr 
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Y  =  po  +  Mi  +  ]tMj  +  ®  • 

j-a 

BirpanHing  xj  in  a  Tayloi^s  aeries  around  1.0  and  deleting  hii^ier  or¬ 
der  terms  yields 


x;  =xi  +  (a-l)x,ln(x,). 
This  is  substituted  into  the  equation 

Y  =  Po  +  i)P|Xi+^XiJ«(xi)  +  e  / 
i=» 


where  a  test  of  the  hypothesis  q  =  PiC^i  -  1)  =  0  tests  the  need  to 
transform.  This  test  is  performed  to  determine  whether  a  trans¬ 
formation  is  required.  If  q  is  significant,  a  transformation  is  per¬ 
formed.  The  indicated  transformation  is  q/pi  +  1 .  Since  this  might 
not  be  a  very  tidy  value,  the  closest  value  in  the  set  (-2,  -1,  -0.5,  0, 
0.5, 1,  2}  was  used.  Each  X  variable  is  considered  for  transformation 
witii  aU  of  the  other  X  variables  in  their  original  scale.  This  ensures 
that  the  transformations  for  the  Xs  are  not  affected  by  the  order  in 
which  they  are  considered. 

THE  RESULTS  OF  THE  MODEL  SEARCH 

In  this  discussion  we  describe  these  four  model  selection  steps  and 
report  on  the  results  of  applying  riiem  to  the  600-part  sample  de¬ 
scribed  in  Section  4. 

The  first  procedure  applied  was  Mallow’s  variable  selection 
method.  The  various  subsets  and  their  prevalence  in  the  three  groups 
are  shown  in  Table  B.l.  In  the  table,  a  zero  indicates  the  variable  is 
not  in  the  model,  and  a  one  indicates  that  it  is. 

Overall,  the  most  frequently  selected  model  is  one  with  only  the  pre¬ 
vious  period’s  demands  included,  althou^  the  flying  hour  and  previ¬ 
ous  demand  model  is  the  most  frequently  selected  for  Group  1,  the 
hipest  demand  group.  The  ^dng-hour-only  model  ties  for  most  fre¬ 
quent  in  group  2.  It  is  always  unsatisfying  to  use  a  model  that  has 
lagged  demands  as  an  explanatoiy  variable.  This  is  tantamount  to  an 
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Table  B.1 

Rasolis  VariaMe  Sdeetlon  Search 


Explanatoiy  Variablea  in 
die  Model 

fh  ^—1  ^—1 

Group  1 

Group  2 

Group  3 

0 

0 

0 

14 

39 

48 

0 

0 

1 

42 

47 

50 

0 

1 

0 

13 

21 

29 

1 

0 

0 

29 

47 

34 

1 

0 

1 

49 

14 

11 

1 

1 

0 

7 

7 

11 

1 

1 

1 

16 

4 

5 

admission  that  we  don’t  really  understand  the  process.  It  is  also  diffi¬ 
cult  to  use  this  type  of  model  to  estimate  counter-factuals.  Questions 
of  the  form,  *What  if  we  fly  more?”  often  need  to  be  addressed. 
Despite  these  caveats,  the  r^tilts  in  the  Table  B.2  suggest  a  model 
with  both  flying  hours  and  lagged  demands. 

It  is  worth  noting  the  firequency  with  which  outliers  were  rejected  by 
this  automated  process.  Table  B.2  gives  the  counts. 

These  rejections  could  be  the  result  of  any  niunber  of  problems  with 
the  data  or  the  modeling.  It  is  useful  to  set  observations  aside  in  this 
manner  in  an  effort  to  capture  the  underlying  structure  of  the  data. 
We  do  not  advocate  discarding  outliers  during  our  predictive  evalua¬ 
tion  of  methods.  It  is  of  tittle  solace  to  know  that  the  30  important 
parts  you  needed  but  did  not  have  were  merely  ‘statistical  outliers.” 

After  variable  selection  we  used  ffie  Box-Cox  procedure  to  explore  the 
need  for  a  transformation  of  the  demands.  The  results  of  this  exami¬ 
nation  are  summarized  in  Table  B.3. 

Table  B.2 

Fractiw  of  Outlier  Belectioiis 

Qronp  1  Oroop  2  Gnmp  3 

37/200  6(V200  67/200 
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Table 

Indieeted  Power  Traaeftwiaetioa 


Group  -2  -1  -0.8  0  0.5  1  2 


1 

0 

2 

10 

26 

88 

66 

8 

2 

13 

10 

18 

68 

80 

11 

0 

3 

13 

14 

22 

58 

72 

21 

0 

The  square  root  transformation  is  the  most  commonly  indicated 
transformation  in  each  oi  the  three  groups.  This  unanimity  of  results 
is  encouraging. 

The  final  model  improvement  that  we  will  consider  is  transformation 
oi  the  explanatory  variaUes.  Table  B.4  summarizes  these  results. 

The  *Omit”  category  indicates  that  the  model  selection  did  not  include 
that  variable  in  the  final  model.  These  results  indicate  no  need  to 
transform  the  explanatory  variables. 

CRITIQUE  AND  SUMMARY 

The  variable  selection  portion  of  this  model  selection  exercise  oc¬ 
curred  before  the  transformation  of  the  demands  was  explored.  Since 
the  variable  selection  was  performed  on  the  untransformed  values, 
there  is  some  iincertainty  as  to  whether  the  results  would  hold  up  if 
the  variable  selection  had  been  done  with  the  square  root  of  demands. 

TaUeB.4 

TVansformations  of  the  Explanatory  VariaMes 


Indicated  Transformation 


Group 

Variable 

Omit 

-2 

-1 

-0.6 

0 

0.5 

1 

2 

1 

fh 

99 

5 

1 

2 

2 

4 

79 

8 

1 

134 

8 

0 

0 

3 

1 

52 

2 

1 

D-1 

63 

5 

5 

8 

8 

1 

108 

2 

2 

fh 

128 

2 

1 

0 

2 

1 

62 

4 

2 

147 

0 

2 

2 

2 

1 

39 

7 

2 

D-l 

114 

6 

5 

8 

0 

0 

66 

2 

3 

fh 

139 

2 

0 

3 

6 

4 

45 

2 

3 

fl*-! 

143 

0 

1 

4 

1 

1 

41 

9 

3 

P-1 

122 

5 

3 

0 

8 

0 

61 

1 

94 


Reninning  this  analysis  with  the  demands  transformed  before  begin* 
ning  leaves  the  variable  selection  and  explanatory  variable  transfin*- 
mation  results  virtually  unchanged 

The  strong  indication  that  the  premier  transformation  of  demands  is 
the  square  root  is  reassuring.  This  is  a  standard  transformation  for 
discrete  data  of  this  type.  The  inclusion  of  demands  from  previous 
periods  in  the  model  is  discouraging  and  may  be  an  indication  that 
the  model  omits  important  explanatory  variables.  However,  these  are 
the  explanatory  variaUes  that  are  availaUe  to  us  and  we  must  live 
with  them.  Lagged  demands  are  unsatisfying  predictors  but  may  be 
useful  predictors. 

The  deletion  of  lagged  flying  hours  from  the  models  should  be  taken 
with  a  grain  ai  salt  as  welL  typically,  fljdng  homrs  do  not  change 
much  from  quarter  to  quarter.  This  strong  correlation  between  adja¬ 
cent  quarters  could  easily  be  responsible  for  the  deletion  of  lagged 
flying  hours  from  the  model.  If  scenarios  with  erratic  flying  hour  pro¬ 
grams  are  considered,  more  examination  of  the  predictive  usefulness 
of  this  variable  would  be  in  order. 
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